Research On Continuous Chinese Sign Language Recognition System Based On Hidden Markov Model And Level Building Algorithm

Recently, the human-computer interaction technology has been widely used in daily life. As one manner of human-computer interaction, sign language recognition technology translate the gesture of the deaf to the word, which improves the communication between the deaf and the normal. Since the continuous sign language is the main language used by the deaf, it is more significant to build the continuous sign language recognition system.In this thesis, the research on the continuous sign language recognition includes two aspects:(1) how to build more efficient sign model and movement epenthesis model; (2) how to split the continuous sign into isolated words and movement epenthesis. The main contents of this paper are:1. Firstly, weighted hidden Markov model is employed to train sign model, where training samples are set with different weights. Secondly, the constraint of the trajectory length is embedded into hidden Markov model, which can improve the recognition performance. Thirdly, the threshold model of hidden Markov model is utilized to generate the movement epenthesis model in this paper.2. Combining hidden Markov model and Level Building algorithm to accomplish the recognition task of continuous sign language. Firstly, the problem of finding optimal segments is translated to the problem of finding the optimal sentence labeling probability, which can be solved by the Level Building algorithm. Secondly, in order to reduce the searching time and improve the searching accuracy, the searching window based on sign frame length and the searching path based on the n-gram language model are embedded into Level Building algorithm. Lastly, a fast hidden Markov model is proposed to reduce the time complexity of Level Building algorithm based on hidden Markov model.3. In this thesis, we build a Chinese continuous sign language system using Kinect as the input equipment, which achieves a 90% recognition accuracy testing on a database of 44 different sentences composed of 66 signs.