Researches On Vision-based Bare-hand Sign Language Recognition

With the improvement of computer performance and the increasing understanding with computer knowledge, the more convenient and the better ways of communicating with computers are required higher and higher. Therefore, multi-modal human-computer interaction with intuitive, natural and friendly is necessary.In daily life, the natural languages which are used by people on communication are spoken language and written language. In addition, there are gestures, facial expressions and other ways to help people to communicate in the case of face to face situation. Sign language recognition can not only provide help for the deaf, but also to promote the development of human-computer interaction research.This paper has researched two important modules in the field of sign language recognition in the two modules have been applied research, there are hand region tracking and feature extraction. In this paper, the main contributions are as follows:(1) Build target template. Convert the first frame of the video image from the RGB color space to the HSV color space, then gain the skin template image via detecting skin color and automatically gaining the region of interest, and establish the histogram of the H component This will solve the semi-automatic problem of continuous adaptive mean shift (CamShift) algorithm, and it is the foundation of the hand region tracking.(2) Hand region tracking. This part mainly work on the application of continuous adaptive mean shift (CamShift) algorithm in sign language video environment, and achieve continuous tracking of hand region in sign language video environment. Retaining the advantage of CamShift algorithm, improve it's application, so that it can effectively automatic tracking.And experimentally proved the validity of the method.(3) Gesture feature extraction. This part mainly work on extraction and analysis of the gesture features and training and recognition of the gesture features. This paper has extracted the features of ellipse fitting of the hand area and the features of features of edge orientation histogram of region of interesting. This paper regards features of ellipse fitting as the location and hand area features; and regards features of edge orientation histogram as the features of the shape of the hand. Finally, train and recognize the features we have extraced with Hidden Markov Model (HMM). In the selected word sets, the recogniton.rate is 69.6%; if regarding the similar gesture words as one gesture word, the recognition can arrive 81.5%.