Research On Facial Landmark Detection Based On Convolutional Feature

Facial landmark detection is obtaining face shape by locating pre-defined facial key point on human face automatically.Facial landmark detection is critical to face image analysis It has been widely used in face recognition,animation,tracking,expression analysis,3D face modeling,and has attracted much researchers' attention.At present,in constrained environment,facial landmark detection has achieved satisfactory result.However,in unconstrained environment,the performance is far from satisfaction due to extreme illumination,pose,expression and occlusion variation,which leads to nonlinear face appearance transformation and variation.Therefore,the existing models perform badly in facial landmark detection and other related tasks,and can't meet the requirement of practical applications.In this paper,we study the facial landmark detection problem in unconstrained environment and propose our improved algorithms.The main contributions are summarized as follows:(1)In unconstrained environment,face appearance transformation is highly nonlinear.But the existing feature extraction techniques can't express the transformation accurately.To deal with this problem,we propose a robust facial landmark localization method based on deep convolutional feature and extreme learning machine.To do this,we design and train a deep convolutional neural network to extract global convolutional feature,which contains rich spatial and semantic information.Then,instead of using the regressor embedded in convolutional network,we introduce the robust extreme learning machine to learn a mapping function from feature space to shape space.Finally,we locating facial landmark position by fusing multiple predictions.The experimental results show that the convolutional feature which contains spatial and semantic information can highlight the general pattern of complex face appearance,and performs better for localization task.The extreme learning machine learns a strong mapping function from the feature space to the shape space.The fusion of multi-scale prediction can further improve the detection accuracy.(2)In unconstrained environment,different face appearances and shapes vary a lot.But the existing cascade shape regression models are sensitive to shape initialization and adopts independent local feature which ignores shape constraint information.To deal with this problem,we propose an improved cascade shape regression model for facial landmark detection.To do this,in the cascade framework,we directly estimate the shape as initialization instead of using the hand-crafted initialization in first stage.In the last two stages,we design and train a multi-objective and shallow convolutional neural network to extract convolutional feature,which contains locally related information.Then,we modified the optimization method of the extreme learning machine to improve generalization.Finally,we gradually refine the landmark result based on the cascade framework.The experimental results show that the learned initial face shape is robust,the locally related convolutional feature can implement shape constraint and improve detection accuracy.In addition,our proposed cascade model extends the existing cascade regression model by employing different kinds of features in each stage.