For Large Vocabulary Continuous Sign Language Recognition System Realization

The aim of research on sign language recognition is to enable the communication between the hearing impaired and hearing-enabled to be free and to enhance the capacity of body language understanding of computers. As the sign language is not used widely, little research on it is held, and the research history of sign language recognition is only about ten years. There are many challenges in sign language recognition, which include: How to realize the recognition of large vocabulary sign language? What is the sign language recognition approach with scalability about the vocabularies? What are the smallest recognition units? How to solve the movement between two signs? How to solve problems for the signer independent sign language recognition? The solutions of the five problems mentioned above are very important to sign language recognition. Only four preceding problems are investigated in this paper. Based on this, a signer dependent large vocabulary sign language recognition system was designed and implemented.Firstly, the comparisons between multi-stream CHMM, multi-stream SCHMM, multi-DHMM and HMM based on banding technique are done here, and the last one is chosen as the kernel technique of the large vocabulary sign language recognition system.Secondly, the siga language recognition approach with scalability about the vocabularies is proposed based on the stream state tying. In order to improve the performance of the system, some useful ideas are employed, including building HMMs which can automatically estimate the numbers of the states based on dynamic programming, modifying the transferring probability, fast matching and using the estimating parameter of transferring between signs in search algorithm. The approach was test on the largest scale hand gesture database (5100 hand gestures) in the world. The experimental results have shown that this approach is very efficient.Thirdly, through analysis, about 2400 subwords are defined for CSL. A continuous sign language recognition approach based on subwords is proposed. Tree-structured network, the forward index table and N-Best are used. The method is compared to that based on the sign. At last, some methods to find the smaller subwords are studied.