A Study Of The Articulations Based On Ultrasound And EGG

With the development of the speech,researchers have a strong interest in the pronunciation organs,and the research on ultrasound images has also made great achievements.Ultrasound can capture the complex motion process of speakers' tongue well when the speaker is pronouncing.The motion data of tongue ultrasound has important significance and wide application for studying the pronunciation process.Through the close mapping relationship between ultrasound data,electroglottography data and speech to study the pronunciation organ motion and speech synthesis.Silent speech synthesis research can effectively help patients with unvoiced or pronunciation disorders to communicate effectively.The speech synthesis system proposed in this paper uses ultrasound to capture the motion data of the tongue,and uses the electroglottography to record the glottal data.Firstly,the tongue ultrasound and electroglottography(EGG)synchronization data are recorded.And through the double threshold method to synchronize the ultrasound images,electroglottography,audio data,then delete the mute section.Extracting the acoustic features of electroglottography and original audio.Through deep neural network to study the features of the vocal organs and the audio.Through the training of the mapping model,the Chinese silent speech synthesis of continuous speech is performed only through the vocal organ motion data.The experimental results show that the speech synthesis method proposed in this paper is the mapping from ultrasound images to Chinese sentence,not only the single phoneme and achieved good results.The internal relationship between the articulatory movements and corresponding speech is also studied in the experimental process.