Research On Sign Language Recognition Algorithm Based On Deep Learning

Computer vision has made enormous progress because of the rapid development of computer.As an important branch of computer vision,Sign language recognition has attracted more and more attention.The main research trends of sign language recognition are the classification,recognition of isolated words and the recognition of continuous words.Thus,there are broad application prospects and important research significance in Sign language recognition in the fields of human-computer interaction,gesture controls and sign language teaching.At present,there are several bottlenecks in the research of sign language recognition:(1)The acquisition of sign language video is difficult.Generally,the accuracy of sign language video is poor;the characters are changeable and the background noise is large.Deep video captured by Kinect depth camera solves some of the problems and adds depth information,but its expensive problem prevents it from being used on a large scale.(2)Sign language gestures have problems of occlusion and fast change,which makes it hard to extract sign language features.(3)Sign language is a coherent action based on time series,but it has many transitional frames between key frames.These transitional frames greatly interfere with the recognition of sign language.(4)The sign language of continuous vocabulary is more coherent and there is no obvious fixed separation gesture,which challenges the segmentation of continuous vocabulary into isolated words.This essay took 4 different ways to research the method to solve the problem.(1)3d modeling software 3DMax was used for modeling of sign language gestures,rendering sign language video as a data set,and data enhancement and noise removal were performed.This method can effectively reduce the influence of background noise on sign language recognition,and can also render multiple videos from different angles to reduce occlusion.(2)Compared with manually designed features,the automatic extraction of sign language features by neural network model has a better identification effect.In this paper,k-means algorithm is used to remove the redundant frames in video of sign language and extract the representative key frames,so as to obtain the characteristics of high discrimination.(3)In this paper,we used LSTM(long-short time memory)model and 3D convolutional neural network to classify isolated words.Compared with 2D neural network model,LSTM model and 3D convolutional neural network model can extract the information of time series in videos,thus achieving better classification effect of sign language video.(4)For continuous sign language recognition,firstly,the continuous sign language video is clustered to filter out the transition actions or gestures in the continuous sign language,then the key frame video is decomposed by sliding window,and the single sign language vocabulary is calculated based on the probability value calculation algorithm,and then the sign language statement is formed by screening.In conclusion,this paper studies the classification features and continuous vocabulary of sign language video in the framework of conventional computer vision research.The experimental results show that the sign language recognition method in this paper has a high recognition rate for isolated words,and the recognition of continuous words also proves its effectiveness.