Research On Facial Landmark Detection Based On Deep Learning

Facial landmark detection,also known as facial alignment,aims to predict the coordinates of unconnected key points of semantic importance,which typically lie in the areas with dramatic changes,such as eyebrows,eyes,nose,mouth and contour.This technique is one of the pivotal techniques for many facial analysis tasks,such as face animation,face recognition,face expression analysis,3D face modeling,age prediction and so on,which has wide commercial value and profound academic value.With the popularization of this technique in daily applications,it suffers from many challenges in complex and various application scenarios.The occlusion problem is a main obstacle to locate the facial landmarks accurately.Many existing methods perform well for near frontal and untainted face images,while the performances degrade severely if faces undergo occlusions.This thesis analyses the main reasons that the current methods are not robust against occluded facial images,proposes two deep learning-based approaches for facial landmark detection.The main research work and innovations of this thesis are summarized as follows:1.A novel branched convolutional neural networks incorporated with Jacobian deep regression framework(BCNN-JDR)is proposed to alleviate the impact of the imbalance errors problem among facial components during training due to occlusions and improves the detection accuracy.The proposed framework firstly exploits the branched convolutional neural networks(BCNN)as the robust initializer to estimate reliable initial shape.By virtue of the componentaware branches mechanism,BCNN can effectively alleviate the imbalance errors problem among facial components.Following the BCNN,a sequence of refinement stages are cascaded to refine the initial shape within a narrow range.In each refinement stage,the local texture information is adopted to fit the facial local nonlinear variation.Moreover,our entire framework is jointly optimized via the Jacobian deep regression optimization strategy in an end-to-end manner.Jacobian deep regression optimization strategy has an ability to backward propagate the training error of the last stage to all previous stages,which implements a global optimization approach to our proposed framework.Experimental results on benchmark datasets demonstrate that the proposed BCNN-JDR is robust against uncontrolled conditions and outperforms the state-of-the-art approaches.2.The occlusion-adaptive deep network(ODN)is proposed to distillate occluded regions of face images in order to suppress the impact of occlusions and improve the detection accuracy.The proposed ODN consists of three modules: geometry-aware module,distillation module,and low-rank learning module.First,to model occlusion,the distillation module is used to infer the occlusion probability map based on high-level features,which serves as the adaptive weight map on high-level features to reduce the impact of occlusion and obtain clean feature representation.Obviously,the clean feature representation cannot represent the holistic face due to the missing semantic features.To obtain exhaustive and complete face feature representation,low-rank learning module is proposed to recover the missing features via learning a shared structural matrix.To assist the low-rank learning module to recover lost features,the geometryaware module is leveraged to excavate facial geometric characteristics so that low-rank module can take advantage of geometric information to better recover lost features.The experimental results show that the proposed ODN is embedded into AlexNet,VGGNet and ResNet to obtain the significant performance improvements.Meanwhile,relying on the synergistic effect of three modules,the proposed ODN achieves better performance in comparison to state-of-theart methods on challenging benchmark datasets.