Researches On Fusion Method Based On Multi-Channel Information Of Sign Language

Sign language, as a kind of most structured gesture, is regarded as an indispensable means of everyday communication for the deaf. In common, Sign language is usually expressed through gestures, facial expressions, head pose and any other means. Moreover, relative to the common people's body language, Sign language is more regularly, and the semantics of sign language is more abundant, so it is considered as a research platform for human-computer interaction.After analysis the current research status, it is found that the researches mainly focus on the hand information processing, module creating and recognition, but seldom on the hand language is a body language based on hand style, arm movement, supported by the facial expression, lips acting and other body posture to deliver conception. The hand-gesture and facial expression play a key point at sign language recognition. And the experimental results show that when there is only gesture without expressions, the content can be understood by people no more than 60%. Therefore, this paper fuses multiple channels of information of the sign language in the sign language recognition process, view to improving the efficiency of sign language recognition. In detail, the main research can be described as follows:1. Training and recognizing the characters of gesture and facial expression respectively using the technology of Hidden Markov Model (HMM);2. Realize the Bayes estimation theory algorithm. And apply it to the fusion of information of sign language. Getting the prior probability using the prior knowledge of the system and the result of the recognition of each channel. And regards the matching probability of each channel as the conditional probability to fuse on the decision-making, to obtain the recognition result of fusion;3. Realize the D-S evidence theory algorithm. And apply it to the fusion of information of sign language. Considering the facial expression recognition matching probability and the gesture recognition matching probability as two separate data channels, fuses the two data channels using the D-S evidence theory on the decision-making to get the fusion result;4. Designed the software architecture for the fusion system of multi-channel information of sign language, and realized the prototype system with more universally and more expansively.