Research Of Chinese Pronunciation Visualization Technology Based On A Physiological Articulatory Model

Pronunciation process visualization refers to displaying the movement of thearticulators during pronunciation process vividly. Pronunciation process visualizationcan be applied in many fields, for example in the field of language learning.Spreading Chinese language worldwide is not only in line with national strategy, butalso an important way to strengthen the national “soft power”. NOCFL (ChinaNational Office for Teaching Chinese as a Foreign Languge) has established a numberof Confucius Institutes in the world. Yet it is still difficult to meet the needs of morethan40million overseas Chinese language learners. A great deal of Chinese languageteachers is still needed. In addition, in the field of speech therapy, there are tens ofmillions of people with speech disorders. The technology of pronounciation processvisualization can help the above patients correct their pronouciations better throughvisual feedback.For the above background, this paper presents a method of using Chineseelectromagnetic Articulographic (EMA) data to drive three-dimensional (3D)physiological model to achieve the method of Chinese pronunciation processvisualization.To begin with, the data were collected. On the one hand, EMA database covering30Chinese phonemes was established, and then a software for data annotation wasdeveloped to label EMA database in phoneme level. The acoustic parameters andarticulatory parameters were carried out to refine data. On the other hand, for the3Dand physiological pronunciation model, nearly150,000muscle combination fileswere designed in order to cover the entire pronunciation space. These files were inputto the3D physiological model and then the corresponding articulatory data weregained.Secondly, before mapping from the EMA data to3D model data, the Thin-plateSpline method was used for adjusting the coordinate system. The EMA data ofChinese neutral vowel [] and the data of3D model at its resting state were comparedto obtain the mapping equation. Meanwhile, in order to verify the preciseness of themethod, an experiment was done to find the relation between the front and rearsections of the tongue. It was proved that applying the information of the tongue’sfront part to analyze is feasible. Besides, to verify the accuracy of the method, by focusing on the seven vowels [a],[],[],[i],[o],[u],[y], their best-matched3Dmodels and the real MRI data were compared. The result showed that an average errorbetween the two was0.21cm, which proved the accuracy of our method.Finally, the visual presentation system at the levels of phoneme and syllable wasprovided. In order to facilitate the users to use our visualization system for verballearning and correction, a web version of the visualization system was developedbased on Struts+JSP.