3d Face Lip Animation Voice-driven Research

Voice-driven lip animation are areas of facial animation technology, and a realistic visual speech synthesis technology has become a research focus in the areas of man-machine interfaces, virtual anchor and artificial intelligence, and it can be used in video teaching , film animation and virtual communities etc.This thesis mainly studies voice-driven lip animation technology based on three- dimensional grid model, focusing on two key issues, (1) visualization of specific human voice and lip synchronization. The method based on data-driven lip synthesis can obtain the true nature of the lip animation; (2) non-specific human voice lip-sync animation. The method adopts BP neural network to research voice-driven lip animation. This method has low computational, is easy to be able to achieve better voice lip animation.This thesis has the following main aspects:1. Collected the voice datas from various people, in order to establish voice database; This thesis studies and analysis the pronunciation of Chinese characters and pronunciation rules, analyzing lip movements states so that a similar status to the state of lip movements are classified, in order to establish mouth-type library.2. Based on specific human visual speech synthesis algorithm. this thesis has classified and clustered lip samples by extracting lip feature parameters, to form the basic categories. Meanwhile, after voice signal for sub-frame processing , extracting the voice characteristic parameters (MFFC), to establish the voice frame and the lip-type mapping, and then establishing the training model to train sample data, in order to finish the final synthesis and lip synchronized with speech frame . After smoothing treatment, the lip frames has achieved a true animation.3. Non-specific person's voice-driven lip animation studies. Based on characteristics of Chinese syllables and the mapping relations between acoustic vowel and the lips types, this thesis builds a three-layer BP network model for the learning and the training. In real-time voice-driven lip animation process, firstly it deals with endpoint detection of speech signals, to correspond the noise segment and the non-segmental directly to the lips closed state, and then extracts the underlying characteristics of an effective voice parameters, after BP network model processing, to generate the lips state frame that is corresponding to real-time voice frame, in order to display the lip animation.This thesis is about the Chinese speech-driven facial lip animation, firstly it extracts feature of speech parameters( MFFC), to establish the mapping between the speech frames and the lip-frame in a time sequence in order to drive the lip movement; Finally, the thesis show voice-driven lip animation in the three-dimensional mesh face model, in order to achieve the true nature of animation effects. The experiment results shows that the proposed algorithm is effective and have satisfactory operating efficiency.