Research On Language Recognition Technology Based On Lip Visual Features

With the rapid development of computer technology and the popularity of smart devices, people have increasing demands of the man-machine interactions. Among all the ways of man-machine interactions, it is convenient and efficient to use language to interact with machines. The performance of traditional speech recognition based on audio is not satisfied in a noisy environment. For this reason, this paper do some research for the language classification and recognition based only on the visual features of lips. The content of the paper mainly includes the lip region segmentation, feature extraction and classification experiment of visual language recognition.In the aspect of lip segmentation, by collecting the images and using OpenCV, face can be detected and the lip region can also be roughly located. Lip pixels can be separated from skin pixels by some operations in Lab color space and HSV color space. At last, lip region is segmented.In the aspect of the lip feature extraction, lip key points are extracted on the basis of lip segmentation. Due to the error of the lip corners of the segmented lip is often large, algorithm of lip corner location is improved. After doing this, lip model is built and lip geometric features are extracted. DCT is used to extract lip pixel features. At last, the geometric features and pixel features are fused together. The experimental results show that the recognition accuracy rate of fused features is higher than either of the geometric features or the pixel features. On the basis of features of signal image, features of image sequences are extracted so that the dynamic information of the lip movement can be take into consideration.In the end, SVM is used to classify and recognise the features because the vocabulary database is small and the performance of SVM is often better than other classifiers under the condition of small data set. Experimental results show that the performance of proposed method of language recognition only based on lip visual features is satisfactory, and the research is of great value.