Research On Sign Language Motion Region Segmentation And Sign Language Recognition Based On Neural Network

Sign language is the main way for hearing-impaired people to communicate,which is mainly expressed by arm and hand movements,body movement trajectory and some subtle facial movements.But for those who are not familiar with sign language,it is very difficult to understand the meaning of sign language.According to the data of the World Health Organization,the number of people suffering from disabled hearing loss has reached 466 million worldwide.Research on sign language recognition is of great significance for the communication between hearing-impaired people and between hearing-impaired people and non hearing-impaired people,which can promote the hearing-impaired and language impaired people to obtain more convenient learning,work and life style.The purpose of sign language recognition is to translate sign language video or image into text or speech output.According to different recognition objects,sign language recognition can be divided into isolated word recognition and continuous sentence recognition.At the same time,the motion region image and the recognition of the motion region can be used in the field of intelligent humancomputer interaction.In recent years,the rapid development of deep learning makes the research in the field of computer vision have another dimension of thinking.Through the experiments of different researchers,it is confirmed that the algorithm based on deep learning has the advantages of strong generalization ability,outstanding modeling ability and rich and effective feature extraction.Based on this,this paper studies the sign language motion region segmentation and sign language recognition by using deep learning method.1.In order to improve the accuracy and intuitiveness of sign language motion region segmentation,a sign language motion region segmentation method based on Deeplab V3 + and Mobile Net V2 is proposed.This algorithm can not only segment the motion region effectively,but also retain the position information of the motion region in the original image,and further get the intuitive trajectory of the motion region.2.The sign language motion region segmentation module is connected with the sign language analysis module to realize the motion region segmentation,tracking,characterization and recognition of sign language RGB image sequence.On the basis of sign language motion region segmentation,a sign language recognition method based on segmentation image is proposed.Because most of the dynamic sign language data sources are in video format,not only the sign language motion area in space is very important,but also the motion trajectory in time domain is of great significance for the recognition results of sign language words.Therefore,the Residual network based on Temporal Shift Module is used as the main network of sign language analysis,which can not only effectively model the sign language in space-time domain,Only the calculation parameters of 2D convolution are retained.Finally,the recognition accuracy of 89.1% is obtained on the large-scale sign language dataset SLR500 with 500 words.3.In order to realize the comprehensive analysis of sign language complete image and motion region image,based on the sign language recognition framework of motion region segmentation,the global image analysis branch based on sign language is added to form an improved sign language analysis TwoStream Networks.Finally,the classification results of the two branches are fused,and the recognition accuracy is 94.7% on SLR500 Dataset.