Research On Facial Landmark Localization Based On Deep Convolutional Neural Network

Face recognition has widely application prospect in the fields of security, access control, human-computer interaction and digital entertainment industry etc. Currently, the uncon-strained face recognition, namely recognizing faces with variations of pose, facial expression, illumination and occlusion, is not completely solved, which hinders the further application of face recognition. Unconstrained facial landmark localization (UFLL) or face alignment, which transforms the face into a canonical form based on facial landmarks, can significantly reduce the recognition difficulity and improve recognition accuracy. UFLL is an important processing step for unconstrained face recognition. The key point of the UFLL is the combi-nation of the local texture feature around each facial landmark and the global shape constraint. Deep learning, especially deep convolutional neural network (DCNN), which can effectively extract the high level semantic features of data through the hierarchical structure, has been successfully applied in many computer vision fields. However, DCNN has been fewer em-ployed in UFLL due to time-consuming training and less of available training data. In this thesis, UFLL is researched based on DCNN with the unconstrained face recognition as the application background. New DCNN structure elements and training methods are explored. The main contents and innovative achievements include:(1) A DCNN based on rectified linear unit (ReLU), padding convolutional layer (PCL) and local response normalization layer (LRN) is constructed for sparse UFLL for the first time. A simple and effective model averaging method based on one data augmen-tation procedure is proposed. A direct regression approach is used to train each DCNN. Multi-level cascade and combination of multiple DCNNs are employed to further im-prove the accuracy of localization. The proposed method solves the problem that pre-existing DCNN cascade algorithms are time-consuming during training. The training speed of our method is five times faster than that of the same scale approaches under the premise of not reducing the accuracy of prediction. It is showed that the ReLU response has the characteristics of distribution sparsity and life time sparsity by careful experiments.(2) A framework of transferring DCNN features for dense UFLL is proposed. DCNN trained on face identification task is fine-tuned on dense UFLL dataset, and embedded as a feature extractor in a locality-principle-regularized cascaded regression framework (LPRCRF). The framework overcomes the limitation of less labeled data with dense facial landmarks for training DCNN for dense UFLL. On the assumption that the sim-ilarity of the source domain and the target domain has a tremendous impact on transfer performance, cascade transfer is proposed to make full use of the characteristics of the cascade framework. The assumption is verified by comparing with direct use and or-dinary fine-tuning. The cascade transfer method achieves competitive accuracy with state-of-the-art methods on the 300-W challenge dataset. A DCNN based on parametric rectified linear unit (PReLU) is proposed, which has verified that the proposed frame-work is adaptable to different model structures.(3) In order to solve the problem of complicated structure and high computation complex-ity of cascade DCNN framework, batch normalization (BN) is employed to inhibit the change of distribution of each layer's inputs during training to train the network more quickly; Multi-task learning, namely joint optimizing sparse UFLL and head pose es-timation etc is applied to train the model under multi supervision signals to learning features with stronger ability of expression. Then the DCNN is transferred to dense UFLL. Finally, a single network is directly employed to predict sparse or dense facial landmarks, which greatly simplifies the algorithm framework and obtains rapid predic-tion speed, and still achieves similar performance with cascaded DCNN methods.(4) Algorithms for face detection, face identification and face verification is implemented based on DCNN. Together with UFLL algorithms proposed in this thesis, all algo-rithms are integrated to construct a whole face recognition system. The usefulness of the proposed UFLL algorithms for face recognition is verified by experiment on benchmarks based on this system. The system provides a foundation platform support to continue to study the DCNN-based face recognition algorithms.