Research On Real-Time Recognition System Of Chinese Sign Language

There are millions of deaf-dumb people in the world communicating by sign language,they cannot communicate through language like ordinary people,and there are many inconveniences in their lives.Thus,it is very meaningful and valuable to develop a real-time recognition system that allows ordinary people to understand their sign language.In this paper,we investigate a realtime Chinese sign language recognition system.The system can recognize deaf-mute Chinese sign language,and output the recognition results to user in real time through text and speech.A Chinese sign language dataset is firstly created.We used RGB cameras and collected 5000 commonly used sign language vocabularies in daily use according to the national common sign language vocabulary.Each vocabulary is demonstrated by 10 different deafmute people,and the entire dataset contains 500,000 video samples.In order to improve the recognition accuracy of the system under the premise of meeting the real-time requirements of the system,we propose a method of 3D-CNN combined with TV-L1 optical flow processing.The collected RGB video stream needs to go through a two-step down-frame processing,and then through TV-L1 optical flow calculation,and finally put into 3D-CNN to extract feature vectors.In addition,we also use the datasets collected in this paper to conduct comparative tests on sign language recognition methods using Hidden Markov Model and Recurrent Neural Network.The results show that the method which RGB video stream is calculated by TV-L1 optical flow,and then put into 3D-CNN to extract features is effective,and achieved 92.6%recognition accuracy on a dataset containing 1,000 vocabularies.Finally,we propose the creation of a complete real-time sign language recognition system,which is composed of a human interaction interface,a motion detection module,a hand and head detection module,and a video acquisition mechanism,etc.Experimental results show that the system has good generalization performance and real-time performance.