Manifold Structure Analysis And Recognition For Signer-Independent Sign Language Data

The aim of research on sign language recognition is to enable the communication between the hearing impaired and hearing-enabled to be free and to enhance the capacity of body language understanding of computers. In order to promote the practicality of sign language systems, Signer-independent sign language recognition is an unavoidable problem that must be solved. At present, the reasons that signer-independent sign language recognition lags behind signer-dependent sign language recognition are the variation and the lack of the training data. The variation of data makes it very difficult to extract effective common features from sign language data. On the other hand, the contradiction between expression of the model and lack of the data has become the bottle-neck to restrict recognition performance in the applications.With reference to the two problems above, this paper introduces manifold idea to traditional hidden Markov model for sign language and mainly deals with the following research work:1. Manifold structure of sign language data is displayed by Isometric mapping. The data set of the same class membership has a kind of embedding manifold structure with certain intrinsic geometric consistency. So such structure itself matches a unique manifold concept. As one of the most popular algorithms to visualize, Isometric mapping is applied to sign language data and proves that the data are contained in a manifold structure. And then we can identify the class indices corresponding to manifold concept with HMM states to find their intrinsic consistency.2. According to the result of sign language data visualized, a TV/HMM model for sign language is proposed. Because manifold concept can learn and reason, the tangent vector approach is integrated into the model to acquire a concise linear representation of the transformations effectively, leaving the class membership unchanged. Besides, the variation of data can be formulized with the description for geometric structure, even if the training data set is small. As a result, not only the variability is expressed, the error rate caused by insufficient training data is also reduced. 3. A sign language recognition system based on TV/HMM is realized. In this system, maximum likelihood estimation is used to learn tangent vector from the training data. At first the optimal parameters and the iteration times are confirmed, and then extensive experiments are conducted to demonstrate the significant improvement of recognition performance contrast to HMM recognition system. Not increasing time complexity remarkably and not requiring a great number of training data, the accuracy of the signer-independent sign language recognition system gets to 72.44% from 70.38%, and the improvement rate reaches 6.96%. Moreover, after the virtual data, which is inward synthesized in mean-shift, are added to the training data, the accuracy rises to 72.94% from 70.56%, and the improvement rate reaches 8.07%.