Researches On Sign Language Recognition Based On Fusing Facial Information

Sign language,as a kind of most structured gesture,is regarded as an indispensable means of everyday communication for the deaf.As one of the most important parts of human-computer interaction,the research and implementation of sign language recognition has important academic value as well as broad application aspect.As the sign language recognition is based on visual system,it can provide more natural and convenient human computer interaction function,so it is raising more and more attention for researchers and has been improved a lot.After analysis the current research status,it is found that the researches mainly focus on the hand information processing,module creating and recognition,but seldom on the hand language is a body language based on hand style,arm movement,supported by the facial expression,lips acting and other body posture to deliver conception.The hand-gesture and facial expression play a key point at sign language recognition.And the experimental results show that when there is only gesture without expressions,the content can be understood by people no more than 60%.So,based on the hand sign language recognition only using hand characteristics,this paper combines the characteristics of facial expressions in sign language recognition,view to improving the efficiency of sign language recognition.In detail,the main research can be described as follows:1.This article choose the color space YCbCr to segment the image with the brightness and chromaticity information,replacing original RGB space method,and receive the more robust segmentation effect.Meanwhile,this paper extract the hands region by use of skin color model and combining regional connection,and then extract the characteristics of hands with the method of ellipse fitting.2.Proposed and implemented that facial expression recognition integrates with the sign language.This article first normalize positioned eye and mouth area in terms of size and gray-scale,selecting a total of 12 key points as the characteristics of extracting expression.Then the face features are extracted through its Gabor transformation. 3.The research on the fusion of hands characteristics and facial characteristics has been studied.Using the strategy of feature-level fusion and decision-level fusion against the fusion of hands characteristics and expression characteristics,and analyzing the experimental results.