Research And Realization Of Lip-reading Recognition System

With lip-reading as an auxiliary method of Automatic Speech Recognition(ASR), combination of acoustic channel and visual channel can attain higher recognition rate than only with acoustic channel. This paper concentrates on key technologic issues such as lip detection, feature extraction and lip-reading recognition.Active Appearance Model (AAM) and Hidden Markov Model (HMM) are combined in this paper. A novel algorithm for lip-reading recognition based on AAM and HMM and lips featured points which can accurately describe lips are proposed. AAM is used to extract feature of lips, HMM used to recognize a feature sequence from AAM. First, spatial distribution characteristic of lips is obtained by training AAM with lips featured points. Second, high-Dimension vector sequence is reduced to one-Dimension scale sequence using K-mean algorithm as observation to train HMM for temporal distribution characteristic of each kind of uttering process. During the recognition, the temporal characteristics of a testing video are analyzed over time by the HMM, and highest score among the likelihood scores provided by HMM estimates the identity of the testing video. Experiments for Chinese digits are made. Experimental results show that the recognition rate of the proposed method is higher, and it has better application prospect.