Design Of Sign Language Recognition System Based On Deep Learning

As a bridge between deaf and mute people and society,sign language plays an extremely important role in the social field,especially for deaf groups to participate in normal communication.The sign language recognition method gradually transitioned from the early stage based on EMG signals to based on images and videos,and the acquisition and processing of EMG signals had problems such as signal interference and high equipment cost,and the use of deep neural networks to analyze and classify sign language images and videos has higher accuracy and stability.With the development of artificial intelligence technology,sign language recognition based on deep learning,as a new human-computer interaction method,has broad application space and research value.Therefore,by extracting the key points of human bones in the image as sign language recognition feature information,this paper designs a sign language recognition model based on Transformer,and develops a sign language recognition application platform to realize the sign language recognition task.Firstly,a sign language recognition scheme is designed,the sign language feature extraction model and sign language interpretation model are deeply studied,and the Continuous SLR100 sign language dataset is expanded to increase the sign language data in complex environments.The sign language video is converted into an image sequence,and the key frames of the sign language video are extracted by the interframe difference method,and the still frames and excessive frames in the sign language process are eliminated,so as to reduce the calculation amount of the model and improve the speed of sign language recognition.Secondly,the upper limb and hand bone keys in the sign language video keyframes are obtained through Mediapipe,the data of the bone key points is enhanced,the bone key coordinates are normalized in two dimensions,and the bone key feature vector is obtained as the input of the sign language translation model,which eliminates the interference of sign language features in the complex background environment and improves the accuracy of sign language recognition.Then,the Transformer sign language interpretation model is optimized,and the sign language feature vector sequence based on bone keys is input into the Transformer to obtain the context time series information in continuous sign language to realize the sign language interpretation function.Comparative experiments were designed to verify the rationality and effectiveness of the sign language recognition method proposed in this paper from four aspects: bone key point extraction,feature normalization method,feature extraction model,and sign language recognition model.Finally,this paper designs a sign language recognition system through Py Qt5 to realize the two-way translation function of sign language and natural language,and establishes a communication platform between deaf and mute people and society,which provides a feasible scheme for the application of sign language recognition technology.