The Research On 3D Visual Speech Expression And Emotional Simulation

Speech visualization is an important branch in the field of speech synthesis research.Realistic 3D face model is the basis and prerequisite for realizing speech expression and emotion simulation animation,it is also a hot research topic which is very important and extremely active in the fields of computer graphics,computer vision,human-computer interaction and so on.It is very difficult for lots of experts and scholars who come abroad or home to synthesize the face mode with realistic vision aspects at present stage.The reasons why it is a challenge research are that the face physiological structure is very complex,the mouth shape is diverse when people speech,the facial expression movement is subtle and the action of the mouth and the complex mapping of the text or sound is with difficult.We build a 3D talking head animation model with text-driven,mouth movement,rhythms and facial expression via combining the knowledge of linguistics and computer graphics based on previous studies.Its advantages are well realism,the realism of verbal expression and emotion simulation,automation,ect,and it has bright application prospects in the film effects,advertising animation,computer games,speech rehabilitation training,virtual teaching and video conferencing and other fields.In this paper,we obtain three-dimensional information from real human facial motion by using the NaturalPoint system of United States based the direct face tracking technology.There are six infrared cameras and companion software named Arena in this system to track the movement of marker points which are on the real person face when speaking.Firstly,we can get the three-dimensional motion position of motion feature points from the three-dimensional motion information.And then map the 3D coordinate points which are processed to face model feature coordination system to drive the virtual 3D face model.The 3D virtual face model is divided into several functional areas which are relatively independent in this paper,and the 3D talking head model is driven by the Dirichlet free deformation algorithm DFFD and the movement of the control points on the rigid rotation transformation.Simulating the interaction influence of different functional areas is through different functional area cross control point.And also making programming in Visual Studio 2010 C/C++,then displaying the result through the open graphical interface OpenGL by adding light,texture mapping and other information.In this paper,speech expression and emotional simulation are driven by different real human facial motion information respectively.The data of speech expression which are from neutral text voice are collected,while the emotional simulation data which come from the statements with emotional are collected by professional performers.At the same time,we collect and process the corresponding database and then regard it as a research direction for audio-video comparison and synchronization,visualization of speech recognition.Next,we verify and evaluate the experiment effects of speech expression and emotional simulation by designing the manual experiment.Finally,we summarize the research work of this paper,analyze and forecast the current shortcomings and problems,meanwhile,look forward to the longing of possible research direction in the future.