Phone-level Based Mispronunciation Automatic Detection And Its Application

With the development of speech technology in recent years, its application has been extended to language learning and pronunciation detection system, computer-aided language learning becomes one of the important research topics. In particular, audio signal, as a kind of media interface, plays a key role in the human-machine interaction.This paper presents a mispronunciation detection system which uses automatic speech recognition to effectively detect the phone-level mispronunciations in the Chinese(L1) learners of English(L2). Our approach extends a target pronunciation lexicon with possible phonetic confusions that may lead to pronunciation errors to generate an extended pronunciation lexicon that contains both target pronunciations for each word and pronunciation variants. The Viterbi decoding is then run with the extended pronunciation lexicon to detect phone-level mispronunciation in learners' speech, and provides corrective feedback to learners.The phonetic confusions are derived from a cross-language phonological comparison based on the theory of language transfer between L1 and L2, and we introduce a data-driven approach by performing automatic phone-level recognition on the learners'speech and analyzing the recognition errors to generate the additional phonetic confusions. The rule-based generation process leads to many implausible mispronunciations, so we present a method to automatically prune and optimize the extended pronunciation lexicon.Experiments based on the speech recordings from 21 Chinese learners of English show that the agreement between automatic mispronunciation detection and human judge is over 86%, the use of extended pronunciation lexicon after pruning can detect phone-level mispronunciation better than using a fully extended pronunciation lexicon.