Research On Key Technology Of Lip-reading System

Lip-reading is a technology beyond to the field of biometrics. Lip-reading is to use the visual information to supplement auditory information to improve the ability of understanding of the computer. Computer lip-reading system is divided into video image acquisition module, lip position module, feature extraction module and lip recognition module. In this paper, the content of these aspects is studied.For lip positioning module, a Gaussian probability model has been established to determine skin pixels and using the projection can determine the approximate location of the face. Based on the distribution of the various organs of the human face, the lip region can be roughly located. Further with Fisher discrimination to distinguish the lip pixels and skin pixels, lip region can be determined accurately.For lip feature extraction module, the local smallest gray method has been used to search the cape points around the mouth. Using geometrical relationship between the cape points around the mouth and the middle points of the upper and lower lips, the two middle points can be determined. the lip geometric features can be used to say by the four key points of the lip region. At the same time, principal component analysis has been used to extract lip pixel features represented by sixty-dimensional matrix.After the lip feature values have been obtained, the video frame sequences need to be cut and classified. The method based on the variance to cut the frame sequences proved to be not effective. there exists the error between the real location and experiment location. In this paper an improved method based on the variance gradient of the feature values has been used to cut the video frame seq uences. The result of the experiment proved the effectiveness and feasibility of this method. In addition, an improved K-mean algorithm has been used to classify the video frame sequences for coding to do preparation for the training and recognition later.Finally, for lip recognition module, the hidden Markov approach has been used to train and recognition samples. Establish a small sub-sample database which is divided into training samples and recognition samples for word ready to recognize. Get the hidden Markov model using Baum-Welch approach with the training samples and recognize the word or words with the Forward Algorithm with the recognition samples. The results show some words have high recognition accuracy which verifies the effectiveness of the methods used in all aspects of the system.