FacialExpression Recognition,Reconstruction And Synthesis

For more than 30 years, facial expression related processing by computer has been an attractive issue in the areas such as computer vision, computer graphics, pattern recognition, etc. It can be widely applied into video conference, movie production, intelligent human computer interface, etc. The work in this dissertation aims at two issues, one is facial expression recognition, the other is face reconstruction and expression synthesis.In this dissertation, the background and the content of our work are described in Chapter 1. In Chapter 2, a technical overview is presented on facial expression recognition (static image based, dynamic video sequence based and audio-visual based), face reconstruction and expression synthesis. An analysis and comparison of the convenient methods on this issue are carried out consequently. The advantages and disadvantages of the conventional methods are discussed and the challenges in these issues are described.The facial expression recognition issue is discussed in three chapters according to different types of data input.In Chapter 3, a GPU based Active Shape Model (GASM) is presented to track and extract facial expression features in static images. The object's profile gradient is strengthened by a GPU based edge filter and a tone mapping operator, which lead to the reduction of iterative times and time cost. Experiments show that GASM is more precise and faster than the conventional ASM. Finally, an SVM-based approach is introduced to perform the expression recognition.The facial expression recognition in dynamic video sequence is introduced in Chapter 4. Firstly, a Kalman prediction based AAM tracking algorithm is presented. The coordinates of the key point m the next frame are estimated from the current one by Kalman filtering. The shape model is updated concurrently. Therefore, the searching space is reduced and the facial feature tracking in the video sequence is faster. Secondly, a video based facial expression recognition model using coupled HMM is proposed. The tracked features are divided into two classes to represent the expression vector and visual speech vector, which keeps the temporal relevancy as well as independence. It shows that the recognition accuracy is enhanced in the novel approach.Based on the work in Chapter 4, the coupled HMM is extended in Chapter 5. A tripled HMM and its training algorithm are proposed. And a tripled Viterbi optimal path searching algorithm is also introduced to make the maximum likelihood decision. Emotional speech is also considered as a contribution to expression recognition in addition to the two visual feature vectors used in Chapter 4. Moreover, a weight parameter is employed to balance the contribution of audio and visual. The whole approach gives a better average recognition accuracy and robustness than the one with only visual based model considered.The issue of the face reconstruction and expression synthesis is discussed in two topics: fast face texture mapping and high resolution face model based expression generation.In Chapter 6, an RBF speed field based real-time interactive texture mapping algorithm is introduced to avoid the complex interactions and great system cost for manipulation on all feature points. The new algorithm presented makes the texture mapping rapid and simple with less cost. A single face image based texture mapping is approached and a series of expressional results are obtained by applying Facial Animation Parameters (FAP).In Chapter 7, firstly, an SVD based 3D face model alignment algorithm is proposed to compute transformation (scale, rotation and translation) between the source and the target model. Secondly, a novel GPU based correspondence algorithm using mesh image with feedback is given, which avoids high computing complexity and great time cost caused by the iterative fittingprocess in the conventional methods. Thirdly, towards 3D facial expression mapping, a Helmholtz-Hodge decomposition based mesh deformation mapping algorithm is presented. The new algorithm can obtain mapped expressional 3D face meshes without any artifacts at a higher speed. Finally, based on this novel expression mapping method, a 3D facial expression synthesizing method based on generic controlling mesh is given. An expressional face mesh synthesis can be performed without target samples.