Tone Recognition Based Mandarin Computer Assisted Language Learning System

Speech is a natural way of human communication. Along with the whole world "Chinese fever" warming up ceaselessly, more and more people learn Mandarin, however, the lack of professional teacher resources becomes more and more critical. While language learning is a long-term project, it needs constant pronunciation practice, so the limited teaching time cannot meet the learning needs. With the rapid development of computer technology and speech recognition technology, it becomes a possibility that we use computer technology to assist language learning. Computer assisted language learning is not restricted by time and place, and then it is flexible and convenient. Computer assisted language learning is an effective complement of traditional teaching way, which has become one of the research hotspots in current speech technologies. At the same time, the research and application of computer assisted Mandarin learning is also conducive to further promote the world promotion and popularization of Mandarin.Pronunciation quality evaluation is one of the key technologies for computer assisted language learning. Automatically evaluating the learners’pronunciation is helpfule for the learners to understand their own pronunciation ability and level, so that they can improve timely, do special exercises, and improve language ability rapidly. Therefore, how to evaluate Mandarin pronunciation objectively is a technical problem to be solved in computer assisted Mandarin learning. Mandarin is a tonal language, whose tone is a kind of unique feature. Through analyzing the characteristics of Mandarin pronunciation and aiming to solve pronunciation problems in Mandarin language learning, we studied the Mandarin pronunciation evaluation technology and realized a computer assisted Mandarin learning evaluating system based on tone recognition. The main contributions of this thesis are as follows:The HMM-based forced alignment for Mandarin utterance segmentation is studied, and the forced alignment based tone model is used to recognize tones. In order to provide relatively accurate syllable segmentation information to the following pronunciation quality evaluation, we pre-process the input speech first, including sampling, quantization and windowing. The39dimensional mel-frequency Cepstral Coefficients are used to extract speech features. Through the HMM forced alignment, syllable timing boundary is obtained in initial speech segmentation. After the training corpus is segmented into syllables by forced alignment, pitch feature of each syllable is extracted at first. The normalized sum magnitude difference square function and Viterbi post-processing algorithms are employed to extract pitch features, which are4dimensional and five tones. Then the segmented training corpus is used to train4-dimensional forced alignment based tone model by using HTK tools. We compared this tone model with the manual segmente tone model in the experiments. It can be found that the recognition rate of forced alignment based tone model is close to the manual segment tone model.A competing model based measurement is proposed to assess Mandarin syllables. According to the initial/final structure of Mandarin syllables, we first define the initial/final structure. Then each syllable would generate an individual competition model set. After that, we put the syllable into its competition model set to calculate the log probabilities of each syllable and find rank of the syllable in competition models. The syllable score is calculated by the size of competition model set and the rank of syllable finally.A simplified competing model based on linguistic knowledge is proposed to assess Mandarin syllables. Because the full competitive model net is too huge, operations tend to affect the system operation speed seriously. To deal with this problem, the competition model measure is improved. The streamlined competition model network based on linguistic knowledge is presented. This improved simplified competition model nets have considerd the Mandarin pronunciation characteristics, linguistics knowledges and pronunciation habits. According to the pronunciation characteristics, the full initial/final competition nets can be final cut into2initial nets and3final nets. Compared with the original competing model network, syllable competition model set generated by the simplified network is reduced by about10times.For the tone evaluation of Mandarin syllables, an integrated tone evaluation method is proposed, which adopts forced alignment based tone model and competing model. Mandarin is a tonal language, so we evaluate tones for Mandarin in this paper. Tone model is trained by a method based on forced alignment. The39-dimensional acoustic model is used to do forced alignment, and syllable boundary is obtained in segmentation. Then the training corpus is segmented into several syllables, and pitch feature of each syllable is extracted. The syllable tone evaluation can be divided into two parts:tonal syllable assessment and tone assessments. In tonal syllable assessment, we take experiments with three competing models:syllable competing models without tone, tonal syllable competing models and tonal initial/final competing models. In tone assessment, we employed tone model and tone competing model. Finally, we integrate tonal syllable score and tone score to get the whole syllable tone score. Experiments show that, this integrated method can give an accurate evaluation result. For the Mandarin sentence score, the weighted average method is used in this paper. We calculate each syllable score in a sentense, and then we get sentence score by calculating the weighted average of syllable scores.A computer assisted Mandarin learning evaluating system based on tone recognition is realized. The system pre-processe the input Mandarin speech and segmente it into syllables by HMM forced alignment, and then evaluates each syllable, gives syllable score, tone score and syllable composite score, finally gives the whole sentence score. The system also gives tips and markes the incorrect pronunciation of syllables.