Applications Of Speech Recognition And Evaluation In Computer-Assisted Mandarin Learning

With Chinese economy fast development, the communication between China and the world becomes more and more frequent in a wide range. Mandarin, the communication tool and culture carrier that lets the foreign country know China. Computer-Assisted language Learning (CALL) plays an important role in modern education technology because of its inherent advantages. Through analyzing traditional language education and existing CALL researcher and tools and aiming to solve the common inaccurate pronunciation problem, we designed a pronunciation evaluation CALL system based on speech recognition, as an attempt in language education field. The speech recognition technique helps the learners have an accurate mandarin pronunciation and calculate the veracity, improving learners' pronunciation. The tone recognition checks their pronunciation tones to make them know whether their tones are correct or wrong as mandarin tone is the most difficult issue confusing foreign learners. Our research is show as follows:1. We design and collect a speech corpora, involving 412 whole Chinese pronunciation, 1319 tonal syllables and 668 isolated words, which is made up of 7 men and 7 women; We also consider reliability, integrality and compatibility;2. We design and propose a tone recognition module, which is based on four levels segmentation, applies autocorrelation and AMDF to extract pitch with dynamic time alignment algorithm. For different Chinese words and expressions or the same Chinese words and expressions that are spoken by diverse persons, frame numbers of the input Chinese words and expressions signals are different. Neural network algorithm is used to tone classification;3. In the module of pronunciation evaluation, we introduce a set of related confidence measures for speech recognition based on local phone posterior probability, scaled likelihood, per-frame entropy and lattice density and present evaluating measurements for pronunciation verify on the levels of phones and sentences separately. Then we address pronunciation evaluation as a prediction problem. We build statistic models and introduce different machine scores, such as HMM log-likelihood, normalized acoustic, intensity, duration and pitch, that can be used as predictor variables via Viterbi force alignment;4. We design and implement the whole interactive mandarin learning software system with speech recognition, speech evaluation and tone recognition. We also design and implement the Web version of our system. Foreign learners can study efficiently from different areas by diverse persons.