Facial Landmark Tracking Via Deep Learning

Facial landmarks may cover the eyebrows,eyes,nose,mouth,and even facial contours and other parts of the face.As facial landmarks contain the semantic information of the human face,it is very important for face recognition,facial expression analysis and pose estimation.So,many scholars devote themselves to it.In recent years,facial landmarks have been attracted in the entertainment industry and other industries,such as face animation and augmented reality.At present,there are many researches on real non-constrained localization methods for static facial landmarks.Although,effective facial landmark tracking algorithms have been proposed,in the real non-constrained environment,it is still very challenging to accurately track facial landmarks due to the influence of rigid and non-rigid factors such as facial posture,expression,illumination and occlusion.Therefore,in the real unconstrained environment,robust long-term facial landmark tracking is still worthy of further study.The main work in this paper mainly reflected in the following aspects:A tracking system is proposed for long-term tracking facial landmarks.The system can be decomposed into three modules: face detection,tracking and facial landmark localization.Thus simplifying the complexity of the system because there is no need to introduce other modules.Each module of the system is relatively independent,with high cohesion and low coupling characteristics,which is conducive to system upgrades.Although the system is simple in structure,the experiment shows that the system has good robustness and accuracy for the facial landmark tracking in the real unconstrained environment.Each module(face detection,tracking and facial landmark localization)in the proposed robust long-term facial landmark tracking system is analyzed and the methods used for the modules are established.The method of face detection is based on the classic Viola-Jones face detection and the best NPD face detection method in unconstrained environment.The tracking method is implemented based on the median flow,which is used in the long-term object tracking framework TLD.It can greatly reduce the complexity of the system instead of using the whole TLD.At the same time,the median flow method can extend the tracking failure detection,which is helpful to improve the robustness of tracking.The method of facial landmark localization introduces the CFAN method based on the deep learning,which can utilize the big data for the model training to improve the accuracy of the facial landmark localization.To achieve long-term facial landmark tracking,tracking verification is introduced.Tracking verification uses the existing face detection method rather than online or additional training a classifier,which reduces the complexity of the system.To further improve the tracking speed of facial landmarks,an adaptive tracking verification method is proposed.It is found that the adaptive tracking verification method can improve the accuracy of large-scale facial landmark tracking.The experimental results show that the proposed robust long-term tracking system for facial landmarks achieve more accurate tracking results than the existing face analysis tool named OpenFace with respective to the facial landmark tracking benchmark database under the real unconstrained environment.In addition,the proposed tracking system does not limit the methods used in the modules.Experiments show that the accuracy of facial landmark tracking can be improved by simply upgrading the modules.