Design And Implementation Of 3D Talking Head System Based On Mandarin Aspiration Airflow Animation Model

The pronunciation of different consonants is very easy to confuse in the perception,which makes it difficult for Mandarin learners to distinguish from the pronunciation of action.Therefore,the study of a speech visualization method based on Navier-Stokes equation makes the identification accuracy of confusing consonants higher,which is important for learners to learn consonants.This thesis analyzed the differences of the airflow parameters.A visualization model of the aspiration airflow has been designed and implemented based on Navier-Stokes equations.Combined with DFFD algorithm,a 3D articulation has been designed to simulate the airflow and the animation of articulatory movements.Based on the 3D articulation,a Mandarin pronunciation teaching software has been designed and achieved.The innovation of this thesis is to simulate the movement process of airflow during Mandarin pronunciation,and display the animation.The 3D articulation which designed and implemented based on the speech visualization method improves the identification accuracy of confusing consonants.The main work of the thesis is:(1)Complete the data acquisition and processing of Mandarin aspiration airflow.Design the general framework of Mandarin aspiration airflow data acquisition,processing and analysis.Select the airflow characteristic variables such as the duration of airflow,the peak velocity of airflow and the amount of airflow as the characteristic parameters for distinguishing the confusing consonants.Complete the syllable segmentation and vocal segmentation based on speech endpoint detection.Complete the statistical analysis of airflow parameters and simulation of aspiration airflow waveform.(2)Complete the modeling and simulation of Mandarin pronunciation airflow.Design the flowchart of the Mandarin aspiration airflow visual modeling.Solve the velocity field and the density field of the airflow based on the Navier-Stokes equation.Solve the external force component in Navier-Stokes equations by using the parameters of Mandarin consonants.Solve the diffuse component by the backward difference method.Solve the horizontal convection component of velocity field and density field under velocity by linear interpolation method.Achieve the aspiration airflow animation model,and complete the simulation of the model.(3)Design and implement the 3D talking head system based on the aspiration airflow modeling.Based on DFFD algorithm and airflow algorithm,drive 3D talking head animation model by EMA data,and drive the airflow model by the airflow data collected by PAS equipment.Realize the function of articulatory movements display,3D talking head model and control points operating,the velocity field and the density field of aspiration airflow display.Compared with the 3D talking head without airflow information,the 3D talking head realize in this thesis has higher identification accuracy for confused consonants.Design eye movement experiment to analyze pronunciation effect of 3D talking head.(4)Design and implement the software for Mandarin pronunciation teaching system.Realize the overall structure and the function design of the software for Mandarin pronunciation teaching system.Complete the design and implementation of user information management module,pronunciation training module based on 3D talking head and listening exercise module.Complete the system module function test.