Research On Facial Landmark Detection Based On Cascaded Regression

Facial landmark detection is a focus issue of computer vision.It aims to locate a set of semantic facial landmarks automatically.The accurate detection of facial landmarks is a premise for carrying out face recognition,3D facial reconstruction,emotion analysis,head pose estimation,and a variety of other facial analysis tasks.In recent years,domestic and foreign researchers have proposed numerous methods about facial landmarks detection.Cascaded Regression(CR)has established itself as one of the most practical and excellent frameworks for facial landmark detection.Many extensions based on CR have been proposed and showed satisfactory performances on the near-frontal faces.However,faces in practical scenes usually undergo challenging cases,including variances of large poses,partial occlusions,rich facial expression,making degraded performance for these facial landmark detection methods.Moreover,most of them detect the facial landmarks adopting static models trained by offline data.When processing streaming data,the static model is retrained from scratch,which is excessively time-consuming and memory-consuming.This dissertation mainly focuses on three challenging problems:the facial landmark detection for an unconstrained facial image,the incremental learning for stream data,video-based facial landmark detection.For these problems,the solutions are proposed in this dissertation based on CR.The main contents and innovative achievements include as follow:(1)Aiming at the problem of facial landmark detection in static images,an efficient multiregressors collaborative optimization approach is proposed based on nonlinear optimization theory.Unlike previous methods that train only one global regressor in one cascade stage,the proposed method aims to solve multiple locally optimal solutions of the objective function in the facial landmark detection problem.In the proposed method,the sample space is divided into several clusters by an iterative algorithm.In each of cluster,samples with similar gradient directions and one separate local-regressor is learned.Then the landmarks of a face image are evaluated by a linear combination of estimations from all cluster-regressors with different weights.The collaborative optimization strategy with considering the inter-complementary and regularization of local regressors greatly enhances detection capability of handling difficult cases with the full range of shapes for cascaded regression.Experimental results validate the detection capability of the unconstrained facial image and achieve superior performance in comparison with the state-of-the-arts.(2)Aiming at the problem of facial landmark detection in stream data,an incremental learning method based on cascaded regression is proposed.The sequential training procedure of conventional CRs largely limits the task of an incremental update.In the proposed method,each cascade's inputs are randomly drawn from a Gaussian mixture distribution thus significantly facilitating the regressors of all cascades to update in parallel.Meanwhile,an Extreme Learning Machine(ELM)is introduced to learn the mapping between facial feature representations and the shape variations quickly.The experimental results of the proposed method validate the effectiveness of model updating.With the increase of learning samples,the performance of the dynamic model has been improved continuously.Besides,this approach is applied to the unconstrained video-based facial landmark detection to reinforce the localization of facial landmarks.(3)Aiming at the problem of facial landmark detection in video,a dynamically cascaded regression framework for facial landmark tracking is proposed to focus on exploiting the informative correlation between previous and current frames.The proposed approach learns the spatial appearance on specific-person statistics from continuous facial frames,which aims to continuously enhance the prediction capability of the tracking model for sequential data.Monte Carlo method is used in the proposed approach to parallelly tune a cascade of regressors based on a dynamic statistical distribution,which guarantees facial landmark tracking in real-time.Meanwhile,we successfully incorporate both the linear and non-linear functions into our parallel cascade framework and introduce Broad Learning algorithm as a solution for them simultaneously.Experimental results on the most popular and large-scale benchmark for facial landmark tracking verify that the proposed method can achieve highly competitive performance in comparison with the state-of-the-arts,particularly in complex situations,such as partial occlusion and rich expressions.