Research On Key Technologies Of Chinese Sign Language Recognition And Translation Based On New Continuous Sign Language Dataset

Sign language is an important way of communication between deaf people.In this era,accessibility has become increasingly important.If we want others to understand our deaf world,we must break down barriers to communication.However,for most people with normal hearing,learning sign language is a very difficult thing,and the learning time and cost are very high.Therefore,many research teams around the world are using deep learning to help research sign language recognition and translation.Most of the existing Chinese sign language datasets are at the lexical level.The videos in the datasets are based on partial frames of sign language words.These data are inefficient for sign language recognition and translation based on deep learning;A small number of sign language datasets are continuous sign language datasets at the sentence level,but the quantity and quality of these datasets cannot meet the needs of deep learning training.Not only to promote the development of sign language recognition and translation technology,but also for the convenience of our deaf people,it is imperative to build a Chinese continuous sign language dataset.Therefore,the main contributions of this thesis are as follows:(1)The Deaf Institute of Tianjin University of Technology is collecting and building a new continuous sign language dataset TUTD.70 sign language learners completed 38150 video segments and 109 phrases in TUTD.The author optimized the continuous sign language dataset by changing the word order to make it easier for deaf friends to comprehend,and 70 sign language speakers donned various colors and repeated recording five times.Also,five announcers annotate each video clip.(2)Based on the traditional Seq2 Seq network,an improved Seq2 Seq network(AS2VT)model based on attention mechanism is proposed using TUTD dataset.Test the performance of the dataset in this paper on the benchmark network and the improved AS2 VT network proposed in this paper.The research in this paper enriches the continuous sign language dataset and is conducive to the development of deep learning sign language recognition.It helps to break communication barriers and make it easier for deaf people to express their ideas.