Research On Head Posture Recognition In Sign Language Video

With the in-depth study of intelligent development of human-computer interaction and computer vision, head posture estimation, as an important research topic,has increasingly aroused people’s exploration interest in recent years.At the same time, head posture recognition has broad application prospects, for example:sign language recognition, intelligent monitoring, fatigue detection,3D games and entertainments. Among them, using head posture to improve the deaf-computer interaction ability and the sign language recognition performance has become an important research focus.Sign language is the main way for deaf people to communicate. Sign language includes not only the hand’s and arm’s movements, but also contains expressions and head posture. The experiments show that when there is only gesture in it, the content can be understood by people no more than60%. The head pose can contain a wealth of meanings and play an important supporting role in a comprehensive understanding of the sign language. This paper has studied the head pose recognition in the video of the sign language. Eventually, it has achieved a head pose recognition system. The specific tasks of study are as follows:1. Face and feature point detection and tracking. Firstly, this paper establishes skin color model to detect faces in the sign language video, tracks them with Camshift algorithm and fits the related parameters of the head with ellipse; secondly, on the basis of the detected face, this paper tracks the facial feature points with the combination of Kalman filtering and Meanshift algorithm, furthermore, according to the geometric model, dynamic threshold binarization and lip color model, we pinpoint the inner eye points and the midpoint of the mouth in each frame of the sign language video.2. Head posture estimation from multiple perspectives. At first, according to the directional angle of the ellipse(the angle between the long axis of the head and the horizontal direction), this paper has estimated the posture change of head within the plane; then, using the distance and position change of facial feature points we acquired, combining with the head parameter information in the front parallel frame of video, this paper has estimated the rotation angle of the head within the space, thus achieving real-time head pose estimation in the sign language video.3. Training and recognition of head movement information. In order to improve the recognition accuracy of the pose categories, this article has extracted the ellipse parameters, edge direction feature and posture feature, divided the selected sign language words into five categories, then trained and recognised the features using HMM. When the selected sign language words were divided into five categories in this paper, the recognition rate can reach78.97%.4. The implementation of head posture recognition system in the sign language video. Based on the program we has studied about the head posture estimation and motion recognition, this paper has designed the software architecture of the system and finally achieved a head posture recognition system.