Research On Chinese Sign Language Recognition For Non-specific People

Sign language is a daily communication method for hearing-impaired people.Sign language recognition translates sign language into text or speech for output,which greatly facilitates the communication between deaf-mute people and other people.At the same time,sign language recognition is an important field in the development of humancomputer interaction system,which has important practical significance in the era of intelligent interaction.Therefore,the research on the Chinese sign language recognition task of non-specific people not only has rich theoretical significance,but also has broad application prospects.Firstly,the current research status of sign language recognition technology at home and abroad is expounded.The theoretical basis of Convolutional Neural Network(CNN)and Recurrent Neural Network(RNN)is introduced.The sign language recognition method based on deep learning is studied.And the sign language recognition system based on deep learning is designed.Secondly,aiming at the problems of small hand shape discrimination and weak spatiotemporal correlation in sign language,this thesis proposes a sign language recognition method based on long short-term spatiotemporal features.The proposed method preprocesses the sign language data,and then introduces the residual structure into a Three Dimension Convolutional Neural Network(3DCNN)to extract the shortterm spatiotemporal features of the sign language sequence,and then uses Convolutional Long Short-Term Memory(ConvLSTM)fully time-series modeling these short-term spatiotemporal features,and finally classify the spatiotemporal features using Softmax classifier.Experimental results show that the proposed method has a very good generalization performance for non-specific groups and can effectively improve the effect of sign language recognition.Thirdly,aiming at the problem of redundant information in the sign language sequence,this thesis introduces the spatiotemporal attention mechanism on the basis of the above sign language recognition framework,and proposes a spatiotemporal attentionbased sign language recognition method.The proposed method spatially constructs a spatial attention model by designing a residual three-dimensional convolutional module,and learns to emphasize the salient content in the feature map in a weakly supervised manner;The temporal attention mechanism is introduced in the ConvLSTM network to identify the most relevant frames in the input video,and achieve different attention to the video frames at different times.Experimental results show that the proposed model with spatiotemporal attention mechanism can further improve the recognition effect,and the recognition rate reaches 95.5%.Finally,a Chinese sign language recognition system is built based on the model proposed in this thesis.The experiments and analysis are conducted on the self-built Chinese sign language data set.This system can realize the recognition of sign language for non-specific people,which meets the actual needs.The effectiveness and feasibility of the method in this thesis are verified through system testing.