A Study On The Expression Synthesis For 3D Face

Facial expression is a fundamental method to convey emotion information between different people,which plays an important role in common communication. Nowadays,computers offer a convenient life to us,and people also wish that computers will be able to communicate with emotions as human being.Especially, the technique for computers to express their expressions is so-called realistic expression synthesis.As a hot topic in computer concerned field,realistic facial expression synthesis has a lot of applications in Film & Television or Game Entertainment,Human-Computer Interfaces,Virtual Reality,Remote Conference or Education,Computer Aided Medical treatment,etc.The techniques for expression synthesis have made a great progress in last years,and many useful models and approaches have been proposed.However,as facial expressions vary in different complex ways,there are still a lot of problems to animate realistic expressions and extend it to different applications.It is still a challenge for researchers to design highly realistic and useful systems for expression synthesis.In order to establish a face independent realistic expression synthesis system for 3D face applications,this paper focuses on 3D face processing,3D face modeling and reconstruction,physical model based expression synthesis,statistical model based expression synthesis,etc.From the physical-based and statistical-based aspect,some common problems in expression synthesis are deeply investigated,and corresponding models and methods are proposed.Based on these studies,a face independent expression synthesis system is designed and completed.The innovative points and main contributions of this paper are summarized as follows.Firstly,an automatic algorithm is presented to achieve accurate point-to-point correspondence between 3D face.In this algorithm,the task to get 3D correspondence is dealt as a 2D fitting problem.At first,several feature points are extracted automatically as the pre-knowledge to obtain an initial pixel-to-pixel correspondence between the two 2D texture images for matching by interpolating.Then a new optical flow method irrelevant to lightings is proposed to refine the correspondence results. This algorithm is able to get accurate correspondence between 3D faces automatically, and is irrelevant to lightings.And these fitted faces are used to construct a statistical model of 3D face,which is the foundation work of this paper.Secondly,by analyzing and summarizing the anatomical knowledge of face,a new self-adapting physical model for expression synthesis is presented.This physical is composed of an improved muscle model,a jaw bone model,a single-layer skin model and a corresponding strain-stress rule of skin.The skin model is compact and the realistic movement of skin issue can be simulated by a specialized strain-stress rule.This physical model has a brief structure and is able to synthesize realistic expressions by lower cost.Also,an automatic fitting program of this model is proposed.A mapping from physical feature to model parameters is achieved in advance.For a new input face,its 3D shape and texture are automatically reconstructed by a Morphable Model,then its model parameters are generated by shape features.This automatic fitting process takes little time and can get accurate fitting result.Thirdly,a bilinear analysis based technique for expression synthesis is proposed, by analyzing abundant 3D face data statistically.After a bilinear analysis for 3D faces, their identity feature and expression feature are independently separated into two linear subspaces.This bilinear model is compact and is rather useful in different 2D or 3D applications.Fourthly,a muscle parameter driven approach for expression synthesis is proposed,to improve the ability of expressing identity and expression in bilinear model.On one hand,the bilinear analysis is applied to segmented 3D face,which gets better reconstruction and synthesis results.On the other hand,a mapping from physical model parameters to statistical expression parameters is learnt.Then by selecting proper muscle parameters,corresponding expression parameters can be generated to synthesize expressions with different contents and intensities.Finally,a manifold analysis based non-linear model is presented to animate facial expressions.In this model,different expressions are projected to different manifolds.And the non-linear changing track between different expression states is trained by a supervised learning progress.This low-dimensional manifold model is able to express the dynamical state of facial expressions.According to the above researches,a 3D face expression synthesis system is presented.In this system,a 3D model for an input 2D face is reconstructed and its realistic expressions are animated by different ways.This system is a common platform for future work.