Research On Lip-reading Technology Based On Video Sequence Segmentation

With the fast popularization of artificial intelligence,the widely used lipreading technology have also been researched thoroughly.Several important aspects of lip-reading are face detection,lip localization,lip feature extraction and lip-reading recognition.Lip-reading algorithms have been widely proposed,but these algorithms are limited to traditional methods of process and algorithm has high redundancy.So on the premise of keeping full mouth information of speaker as much as possible,this dissertation is committed to mouth sequence segmentation and normalization,which can reduce complexity of lip-reading follow-up work and improve accuracy and effectiveness of feature extraction and recognition algorithm.The lip-reading database is the foundation of research on lip reading.The integrity of database is also one of the conditions to ensure the effectiveness and practicability of the following algorithm.Based on the actual needs of the project,this dissertation extends the database on the basis of the original database HITSZ-VSR-B in the laboratory,so as to be used for subsequent test research.In the foundation of detecting face,lip positioning is mainly divided into two parts,which is rough positioning of lips and accurate positioning of lips respectively.In order to locate the lip area accurately,this dissertation proposes an accurate segmentation algorithm of lip region based on YIQ(National Television Standards Committee)and HSV(Hue Saturation Value)two color spaces.After rough positioning of lips,the algorithm makes use of the characteristics of the Q component in YIQ color space and the H component in HSV color space and combined with the results after the processing in two color spaces.After many experiments,it is proved that this method can achieve stable an d accurate segmentation for different lips.In order to reduce the influence of the complexity of feature extraction algorithm on the subsequent recognition results,a method of lip sequence segmentation based on visual static features is designed in this dissertation.On the premise of keeping full mouth information of speaker as much as possible,the lip feature sequence is segmented and normalized.Then,this dissertation uses LDA algorithm to make feature reduction for DCT algorithm,and then HMM method is used to recognize the feature.After many experiments,it shows that the best recognition rate of this feature extraction method is higher than that of PCA and DCT,and the effectiveness of the feature extraction algorithm is proved.