Research On Facial Landmark Localization Based On Cascaded Pose Regression Under Partial Occlusion

Facial landmark localization aims at locating facial landmarks automatically,which is the prerequisite and foundation for analysis of face images.However,it is still a challenging task to locate the facial landmarks under the partial occlusion.Recently,the cascaded pose regression model has attracted increasing attention,due to its superior performance in facial landmark localization.Especially,this model can not only locate landmark but also detect occlusion.It typically begins with an initial shape and then updates the shape and occlusion state from coarse to fine through the trained regressors.It is very sensitive to initial shape,where an improper initialization can severely degrade the performance.Also,the existing algorithms update location and occlusion state only through one-stage cascaded regression,there is no reasonable method to evaluate whether the prediction results are reliable.Especially,in the early stage of regression,the detected landmark is inaccurate,which makes occlusion detection inaccuracy,it will also affect the accuracy of locating landmark.To overcome those problems,three improved models are proposed in this paper.Cascaded pose regression model is very sensitive to initialization,while most traditional methods are based on random initialization,which would severely decrease the performance of regression.Therefore,in this paper,we propose the cascaded pose regression based on texture correlated initialization.By extracting the texture histogram matrixes of all faces,the correlation between the test face and the training face is calculated by the Pearson correlation,then the shapes of the training faces that are most correlated with the testing face are selected as the initialization.The proposed method improves the robustness of the initial shape.The accuracy of landmark localization on the COFW dataset is higher than exsiting algorithms,and the occlusion detection is improved from the 80/42%(Precision/recall)to 80/51.4%.The texture correlated initial shape considers the texture feature but ignore the pose of the face.When occlusion and pose exist simultaneously in a face,the accuracy of localization is still unsatisfactory.To further make the initial shapes more robust to various poses,we propose the cascaded pose regression based on robust initialization.We estimate the pose of the face according to the five fiducial landmarks.Then the pose correlated initial shapes are constructed by the mean face's shape and the face pose.Finally,the texture correlated and the pose correlated initial shapes are joined together as the robust initialization.Instead of taking the median of all predicted results as the final output,the variance is used to determine the reliability of two initialization methods' predictions and take the median of the reliable predictions.The experimental results show that our method obtains a remarkably higher accuracy on facial landmark localization and occlusion detection.Accurate initialization will improve the performance significantly.However,there is no method to assess the reliability of the prediction results in the one-stage cascaded regression model.Also,updating occlusion state simply through regression makes occlusion detection inaccuracy.Therefore,we propose two-stage cascaded pose regression based on sparse reconstruction revise.Firstly,the cascaded pose regression based on robust initialization is used as the first stage to get primary results.Then,according to the primary results,the sparse reconstruction algorithm is used to calculate the reconstruction residual of each landmark,which is used to determine the reliability of the primary results and modify the unreliable results.Besides,according to the non-zero terms in the sparse coefficient,the corresponding dictionary elements are used to recalculate the occlusion probability of each landmark.Finally,the modified results are used as the initialization of the second stage regression to get the final output.The experimental results demonstrate that the proposed scheme achieves better performances than the cascaded pose regression based on robust initialization.In this paper,we propose three improved models to solve that the traditional cascaded pose regression model is sensitive to initialization,one-stage cascaded regression model has no reasonable method to evaluate the prediction results and the accuracy of occlusion detection is unsatisfactory.Experimental results show that the proposed schemes significantly improve the accuracies on both facial landmark localization and occlusion detection on partially occluded face than the state-of-the-art benchmarks.