Research On Sign Language Recognition Algorithm Based On Deep Learning

Sign language is based on the change of gestures,simulating pictograms and syllables to form their specific meanings and expressions.Standard sign language includes five main components:hand shape,position,motion,direction and NMF(expression).Sign language is an important "assistance tool for voiced language" between the hearing impaired,the deaf-mute and the normal people,and it is the main communication tool for the hearing-impaired.It is the existence of sign language that does not limit the living space and development space of the monks because of language.With the continuous development of computer technology,the contactless human-computer interaction has been continuously improved compared with the traditional contact human-computer interaction user experience.People are more and more demanding in the process of human-computer interaction,which is more intelligent,convenient and efficient.The vision-based gesture recognition system enables contactless interactions in space,such as education and teaching,VR/AR,smart home,smart driving,medical fields,and industrial applications that affect human life.Man-machine interaction between sign language is vivid,intuitive and has a strong visual effect.Therefore,in the future,sign language will become a new type of human-computer interaction using universal contactless.The recognition of gestures is generally affected by factors such as skin color,hand shape and posture.The recognition system for constructing gestures has different applicability due to the difference of the extracted feature gesture information and the selected recognition algorithm.This thesis focuses on the research of gesture recognition based on deep neural network model.The specific research work is as follows:1.Use Kinect 2.0 to collect gesture images and videos to build a static sign language database and a sign language video library.The static sign language database consists of two parts:a color sign language image database in a complex background and a depth image sign language database in a simple background,and the depth image in the database is preprocessed and segmented to obtain the target hand type.In this way,it not only overcomes the influence of skin color on the recognition result,but also avoids the influence of light sensitivity,high environmental requirements,and low robustness.It laid the foundation for the subsequent classification of sign language information.2.The gesture image is taken as input,and the convolutional neural network is used to identify the static isolated sign language.In the realization process of static sign language recognition,the experiment of color sign language image recognition in complex background and depth image sign language recognition in simple background is completed.The experimental results show that the recognition accuracy of the depth image dataset is higher than that of the color image dataset.In addition,compared with the traditional descriptor representation of gesture features and classifier design,the convolutional neural network is used for static.Sign language recognition is more efficient and accurate.Finally,based on the trained network model,the real-time sign language recognition of the gesture image taken by Kinect is realized.3.On the basis of completing the sign language recognition in the image,it is proposed to combine the convolutional neural network(CNN)and the long-term short-term memory network(LSTM)to identify the specific sign language video activities.CNN is mainly responsible for extracting input gestures.The feature vector of the data;then,the acquired gesture feature vector is input into the LSTM network according to the time series for gesture recognition,and the recognition accuracy of the video sequence reaches 99.256%.