Study On Sign Language Recognition Based On Sign Language Linguistics And Human Kinematics

With the widespread use of the computer in modern society, traditional human machine interaction (HCI) technologies based on mouse and keyboard show their increasing limitations. Thus, the research on multimodal HCI becomes more and more important in real life. Sign language recognition (SLR), as one of the important research areas of HCI, has spawned more and more interest in HCI society.An ideal SLR system would have a large and general vocabulary, support the user's need of movement to the widest range possible, and conduct real-time recognition precisely and robustly under various complex environments; also the recognition system must meet the needs of native, unspecific users. At present the large vocabulary signer-independent SLR still can not reach the same level as signer-dependent SLR; this gap is mainly caused by data variation and the lack of training samples. Data variation makes it difficult to extract the effective common features from sign language data in signer-independent SLR. Because of the lack of training data in sign language recognition, it is extremely difficult to train any complicated model; this has made it hard for the research on sign language recognition to proceed. So it is important to solve these two issues for Chinese Sign language recognition and other related fields. The data variation and the lack of training samples can be solved effectively from the aspects as follows in this paper:1. The notation sign system considering the sign language linguistics and human kinematics is established according to the characteristic of linguistics and human kinematics of the sign language, while keeping the integrality of the originality data. Then based on the notation sign system and the present status of the lack of the method of sign language data verification, a new verification method of sign language data is presented, which satisfies the characteristic of sign language and is used to verify the effectivity of the generated and transferred sign language data.2. A new data sample generation methos is proposed, which satisfies the characteristic of linguistics and human kinematics of the sign language. In order to make sure the generated data is high quality trend increase, the genetic algorithm and Baldwin effect are used.3. For the problem of data variation, first, the method of using orthogonal Riemann manifold is presented to find the data variation. Then, using the tangent vector of orthogonal Riemann manifold to improve the statistical recognition model, and which is used to solve the problem that the data variation influences sign language recognition result.4. In the presented method of sign language data generation, the"selection"action of genetic algorithm is implemented depending on the present sign language recognition system. In order to insure the validity of the"selection"action of genetic algorithm, the verification method for the robustness of sign language recognition system is presented.