Face Alignment Algorithms Based On Ensemble Of Deep Networks

Face alignment is an essential step in many face analysis tasks,such as,face recognition and expression recognition.In this paper,we propose 68-point detection method based on a parallel ensemble of convolutional neural networks(CNNs)and 5-point detection method based on a sequential ensemble of auto-encoders.The main contents are as follows:We change the structure of AlexNet,a CNN used for image classification,and build FineAlexNet for 68-point detection.Considering the parallel ensemble strategy will improve the performance of the algorithm,we design a CNN ShortNet for 68-point detection with fewer parameters.Neither the initial shape nor the hand-crafted features are needed.The two CNNs could learn features associated with face alignment and the relationship between features and shapes end-to-end.By the parallel ensemble of ShortNet and FineAlexNet,we can train and test them at the same time,not only saving time costs,but also obtaining more accurate estimations.When generating samples for 68-point detection,we add glass occlusion and random occlusion to the face image,and we rotate the original image,aiming to enhance the robustness against difficult images with occlusions and pose variations.In order to reduce the influence of where the detection box is on our method,we crop the face region in the box after translation.The preprocessing above not only makes our method robust,but also increases the number of samples.As for 5-point detection,we propose initial auto-encoder,auto-encoder regressor and sequential ensemble of auto-encoder for face alignment.The initial auto-encoder learns a more suitable initial shape for image than the one chosen from the shape set.The auto-encoder regressor extracts high-level local features from hand-crafted local features of images,then estimates more accurate shape difference.Starting from the initial shape output by initial autoencoder,the sequential ensemble of auto-encoders updates the shape estimation iteratively,with several auto-encoder regressor,until close enough to the ground truth.We conduct experiments and evaluate our method to confirm its accuracy and robustness.When evaluating on LFPW,HELEN and 300-W datasets,the mean errors are 5.41%,5.34%,6.34%with inter-ocular distance,and the mean error is only 10.31%on Challenging Subset,more accurate than some classic algorithms,and proving that the preprocessing makes the method more robust to challenging images.The mean errors are 3.91%,3.84%and 4.56%with inter-pupil distance,which are 1.25%,1.26%and 1.43%with diagonal of the bounding box,close to DAN-Menpo.We evaluate algorithm on CelebA dataset,the mean error with interocular distance decreases from 6.66%to 2.18%stage by stage.