Chinese Speech Synchronized3D Facial Animation

Chinese speech synchronized3D facial animation is an important aspect in the research of natural man-machine interaction. One of the most important speech synthesis advances is re-combining an arbitrary sound with the "Avatar". It is believed that visual synthetic speech will prove more valuable than hearing a synthesized voice. Speech visualization can provide a more nuanced assessment of problems in mental physics and psychology, which is not available in natural language. Moreover, adding visual information can significantly improve intelligibility. Currently, however, there is still not a considerably good method for realizing Chinese speech synchronized3D facial animation which can accord with the habits of Chinese pronunciation thus making the generated avatar look dull and lifeless, which in turn impairs the intelligibility and recognition in man-machine interaction. Therefore, the objective of our research is to explore a new method for speech visualization, and to establish a Chinese-language-based speech synchronized animation synthesis system for avatars, which is applied widely in news broadcasts, dialogue systems, virtual host, virtual meetings, film production,3D games, entertainment fields, etc.In order to meet the characteristics of Chinese pronunciation and satisfy the requirement of the speech visualization technology, namely the natural and continuous lip animation, a speech synchronized3D facial animation system is achieved, which considers sufficiently the habits of Chinese pronunciation. This system consists of three parts:3D facial model; co-articulation model; synchronizing the animation-stream and the speech-stream;In the first part, with the research of the anatomy of the facial movement, we build a3D facial control model based on the muscle model and kinematic geometry model which are controlled by the form of data structure for facial movements. In order to achieve a more realistic effect, we build the tongue, teeth and other models to suit the sound of vocal organs.In the second part, a speech visualization co-articulation model is constructed where the Chinese pronunciation property would be sufficiently considered. Through this method, the inter syllables weighting function of consonant-vowel is used to simulate the effect of co-articulation and lip animation that obey Chinese pronunciation habit.In the third part, to solve the problem of Chinese Speech Synchronized3D facial animation, a method of synchronizing animation-stream and speech-stream is presented. Firstly, Through the Chinese text analysis, we obtain the Chinese visual phonemes; Secondly, through the time locating of Chinese visual phoneme speech, speech and facial animation are combined, which ensures the speech-stream and animation-stream are aligned in the timeline. Finally, we use the interpolation algorithm to generate speech synchronized3D facial animation. This method can improve coherence and rationality of the facial speech animation.According to the above research, a speech synchronized3D facial animation system based on Chinese text driven is established. According to the Chinese text input, the system can, through speech visualization technology, generate Chinese speech synchronized3D facial animation. In order to effectively evaluate the performance of the method, we perform comparisons and analysis of experiments with the principal and objective evaluation method, the results show the synthesized animation is more natural and accords with the habits of Chinese pronunciation.