Study On Facial Landmark Localization

In computer vision and image understanding field, the detection and localization of atarget object within an image is an active topic. In face related researches, facial landmarklocalization act as a key step in face image analysis. Many other related researches, suchas face recognition, pose estimation, expression analysis and face animation are dependon the results of the localization. Therefore facial landmark localization attracts moreand more attention. However, due to many real world effects, including pose variations,expression variations and lighting conditions, the results of facial landmark localizationare not accurate enough. Therefore it is necessary to develop accurate, efficient andcapable methods for facial landmark localization in face related researches. Aiming atthis target, we carried out theoretical studies and practical applications.There are mainly two aspects in facial landmark localization and face shapeextraction: one is the local texture modeling for a single landmark, the other is the globalshape modeling for all landmarks. We implement new methods in the two aspects andachieve good results. In this thesis, the main contributions are summarized as follows:(1) This thesis proposes bidirectional cascaded classifiers for facial landmarkdetection and localization. The overall accuracy of landmark localization depends onthe individual landmark detectors, especially some key landmarks which provide theinitial positions for the other landmarks make the localization algorithms more stable.We exploit the bidirectional cascaded classifier to model the local texture and combinethe feature extraction and pattern classification methods. Here,"bidirection"refers tothe procedure that adopts bootstrap to resample the positive and negative samples in turnsin the training phase. Compared with the traditional methods, the method of this thesisis of the following advantages: First, it can cope with large scale data sets. Second, itcan deal with complicated variations of the positive and negative training samples. Third,no matter in the training stage or the testing stage, the algorithm can rapidly reject largenumbers of simple samples, which brings high efficiency.(2) This thesis proposes the enhanced active shape model to solve the landmarkmissing problem. We introduce a global shape model to integrate the individual landmarkdetectors and take the outputs of the landmark detectors as the final confidence of the landmarks. The landmarks which are detected with a low confidence are considered asmissing landmarks. In this situation, the positions of these landmarks are predicted byother reliable ones together with the global shape model but rather than the outputs of thelandmark detectors themselves. The prediction of the unreliable landmarks is derivedby maximizing an objective function of shape probability analytically. Therefore thelocalization algorithm can solve the missing landmark problem and the accuracy will notbe affected too much when some landmarks are detected with large errors. Using thisstructure, the active shape model is enhanced effectively.(3) A face shape extraction method based on Bayesian inference is proposed in thisthesis. The distributions of individual landmarks and the global shape are representedas the probability distributions and integrated into the Bayesian inference framework.The goal of localizing the landmarks is then transformed to maximize a posteriori. Twotypes of parameters which control the shape, including the geometric parameter and shapeparameter, areoptimizedseparatelyduetotherelationshipbetweentheshapeinstanceandparameters. Gradient ascent method and Gaussian-Newton method are used respectivelyin the optimization procedure. A measurement of probability gradient hints (PGH) isdefined according to the derivation of the algorithm. The face shape is updated iterativelyand driven by PGH to achieve the goals of facial landmark localization and face shapeextraction.(4) We propose a facial landmark localization and face shape extraction methodbased on the face coordinate regression. On the basis of the above mentioned frameworkof landmark detection and shape modeling, we further represent the face shapes in aface coordinate system. The problem of facial landmark localization and face alignmentis fulfilled by corresponding face coordinates in each face image. A facial landmarklocalization method is proposed under this face coordinate system via face coordinateregression. The regression method reveals the face coordinates according to the localappearance. The optimal shape is obtained by iteratively optimizing an objective functionwhich combines the regression error and shape constraints.The above algorithms are verified on several face databases. The experimentalresults validate the effectiveness and feasibility of our methods for facial landmarklocalization.