Research On Fiducial Facial Point Extraction Based On Invariant Constraints

Automatic extraction of fiducial facial points is one of the key steps to face tracking, recognition and animation as well as video communication. In practical application, such as surveillance systems, collected face images captured by cameras are often with great pose variations on uncontrolled conditions, and also affected by expression and illumination. However, extracting the fiducial facial points with great pose variations is always a challenging problem. Pose variations bring nonlinear factors into face images, and traditional methods derived from Active Shape Models (ASM) suffer from significant facial variations especially pose changes. Recent regression based methods become the research focus, it can deal with face images with pose variations on some conditions, but highly rely on the available training sets that cover facial variations as wide as possible.In this paper, we introduce and extend a novel projective invariant, named characteristic number (CN), which enables us to incorporate homography collinearity, cross ratio, and geometrical characteristics given by more (6) points into the extraction. Human faces are highly structured and present common geometrics across age and gender of individuals, for example, four eye corners are collinear. We use CN to derive strong shape priors, which characterize the intrinsic geometries shared by human faces. As CN is a projective invariant, only a few of frontal upright faces as training set can calculate this constraint. We combine these shape priors with simple appearance based constraints, e.g., texture and edge/corner, into a optimization. Thereafter, the solution to facial point extraction can be found by the standard gradient descent. Extensive experiments on facial images from several data sets with great variations demonstrate the superiority of the proposed approach compared with the classical texture-based, shape-based and regression-based methods. Experimental results show that our method can extract fiducial facial points more accurate and apply to deal with the face images with expression, illumination and occlusion on uncontrolled conditions.