Research On Lip Language Recognition Based On Deep Learning

With the rapid development of technology,intelligent human-computer interaction applications are becoming more and more extensive,as one of the most convenient ways of human-computer interaction,speech recognition has been studied by more and more researchers.But in some complicated scenes such as noisy,speech recognition has been greatly affected.It is difficult to meet people's needs.Therefore,lip recognition technology came into being.This technology has broad application prospects in the fields of speech recognition assistance,public safety analysis,animation lip synthesis and face unlocking.Lip language recognition is very challenging in computer vision research.It mainly identifies the corresponding text content according to the dynamic change of the lips when the person speaks in the video.But because different people have different lip appearances,and the dynamics of the lips of different people are different,making lip language visual information diverse.Increased the difficulty of recognition.In response to this problem,and based on the advantages of deep learning algorithms in image recognition,natural language modeling,time series prediction,etc.This paper adopts a lip language recognition method based on convolutional neural network and long short-term memory(LSTM).During the implementation of the method described herein.First,the face is detected by Faster R-CNN.Then position the face to the lips,extract key points of the lips,Then,the feature sequence of the extracted key poin ts is input into the long and short time memory network to extract time series information and semantic information,Finally,the results are predicted by Softmax.This paper evaluates the method in the published OulusVS,GRID,MIRACL-VC and other data sets.The results show,compared to traditional methods,The method identification rate in this paper is at least 20% higher than other traditional methods in GRID,MIRACL-VC and other data sets.It also performs better on most OuluVS datasets than most publi shed methods.