Research On Lip-reading Technology Under Variant Illumination

Lip-reading is a basic technology which has wide application prospect and great research value.Recently,more and more algorithms for lip localization and feature extraction have been proposed,but the study of these algorithms mainly focus on the positive ideal light conditions,and the actual lip reading recognition system will be applied in the environment of variant illumination.Therefore,this paper is devoted to the study of lip-reading technology under variant illumination,to weaken the effects of external light conditions on the lip reading and improve the robustness of the lip localization and lip feature extraction algorithm.Lip-reading database is the footstone for the research of this paper.Therefore,this paper firstly studies and compares the existing lipreading database to draw lessons from thier methods and ideas.On this basis,according to the need of this subject,lip-reading database under variant illumination is established for the follow-up research.In order to accurately locate and segment lip area,this paper designs an lip localization algorithm of three steps.Firstly the Haar-like feature and Ada Boost algorithm is used to locate the face.On this basis,the lip is coarse positioned according to the inherent structural characteristics of human face.Finally in the HSV color space,H component is used to segment the lip accurately.Experimental result shows that the algorithm proposed by this paper can locate the lip accurately under variant lighting conditions.In order to reduce the influence of the external illumination change on the lip feature extraction,the robustness of the algorithm is enhanced by two aspects: the preprocessing chain of illumination normalization and the illumination invariant feature extraction.the preprocessing chain of illumination normalization proposed by this paper is composed of median filter,Gamma correction,multi scale Retinex filter and contrast equalization.The preprocessing algorithm can effectively eliminate some of the light noise.Besides,the traditional LBP feature extraction algorithm is improved and the lip is extracted improved LBP histogram features.This feature has certain illumination invariance,which can further improve the lipreading recognition rate under variant illumination.In this paper,the SVM algorithm is used to identify the lip feature.Aiming at the defect that SVM algorithm can only be used in the two classification,this paper uses one to one strategy to make it identify multiple words.For the the problem of SVM algorithm requiring the input feature vector dimension to fix,this paper designs the lip feature sequence length normalization algorithm to unify the dimension.Finally,the proposed lip feature extraction algorithm is verified by the SVM algorithm under different conditions of both natural and variant lighting conditions.