The Research On Facial Expression Recognition Based On Tracking Feature Points And LES-DSN Network

With the rapid development of computer vision,pattern recognition and other fields,facial expression recognition technology has become a hot spot of research Different from static images,the expression feature extracted based on the video contains more static and dynamic information.The static information is mainly embodied in the extraction of expression features in each image,and the dynamic information is embodied in the changing process of the overall expression formed by the set of multi frame images.In addition,the performance of deep learning in many applications has surpassed the traditional machine learning methods in recent years.How to use deep learning to improve expression recognition rate has become an important research direction.The main contents of this thesis are as follows:First,a feature-based two-stage face tracking algorithm is proposed.The algorithm can not only achieve the rigid head tracking,but also can track the deformed face,that is,it can extract the rotation,translation and facial action parameters in real time.Feature based means that the Candide3 3D face model is fitted to the face in real time by tracking the feature points which are selected from the surface of model.In order to obtain accurate feature point tracking,the algorithm combines image-based and model-based tracking,then uses extended Kalman Filter to solve the 2D-to-3D problem,that is,estimates the parameters of 3D model with 2D feature points coordinates.The experiment shows that the tracking algorithm can track the change of attitude and expression in real time on consumer hardware.Then,the expression recognition method is researched based on deep learning.Based on the research of DSN and ESN,an ES-DSN(Echo State-Deep Stacking Network)for expression recognition is proposed,using the "echo" characteristic of ESN as the basic module of DSN.In view of the fact that the Cohn-Kanade expression database has a single structure and a limited number of training samples,which can't meet the needs of large training samples,the expression database is normalized and expanded by disassembling and combining the expression sequences,which meets the need of facial expression classification.The validity of the extended expression database and the expression recognition ability of ES-DSN are verified by experiments.Finally,the ES-DSN is improved.The Mini-batch Gradient Descent is used to train the original fixed input weight and recurrent weight in the ESN model.On the basis of ES-DSN,the LES-DSN(Learning Echo State-Deep Stacking Network)of learning input weight matrix and recurrent weight matrix is proposed.The influence of the several key parameters of LES-DSN on the accuracy of expression recognition is analyzed by experiments.It is verified that facial expression recognition based on LES-DSN has higher accuracy and robustness through multiple comparison experiments.