Real-time Gesture Recognition In Human Computer Interaction Applications

Gesture is a kind of natural and direct mode of communication in everyday life. As human becomes the more and more central part in human-computer interaction, the research on gesture recognition becomes a focus in this field. Gesture recognition provides the user a means to interact in suppositional environment, which is a kind of human-computer interaction techniques most expected.However, gesture has the characters of multi-mode, multi-meaning and has discrepancy under certain time and space situation; moreover, human hands are complicated transformed objects and there is visual instability, all of which make gesture recognition based on sight become a challengeable multi-subject research goal.There are two kinds of gesture, dynamic gesture which has the definition as movement locus of hands and static gesture which expresses meaning through hand-shape. The part of Real Time Gesture Recognition in this article not only includes static gesture research but also dynamic gesture research. Combined with the subject "Gesture Recognition and Synthesis" endowed by Shanghai natural-science fund, the article conducts certain research on time gesture-recognition and try to let it take over mouse in operation.The article has five parts in gesture-recognition and application course: Time collection in video frequency, Picture pre-transaction, Gesture character extraction and recognition, and Human-computer interaction (drive mouse). In picture collection, we use USB pick -up camera to collect gestures and get 32 bits image containing 320*240 pixels. In gesture pre-transaction, we conducted several operations including turning the RGB color images into gray-scaled images, counting the binary version of the images, smoothing image to reduce the noise in the images and extracting the outline of the hand.In the part of gesture feature extracting, we applied the geometry features of the image, analyzed the images' distribution features in space domain and rules of the pixel histogram. Finally we selected several features including bothparcel and holistic features which can divide the gesture most well. In the recognition part, the paper adopts decision-tree based on rejection strategy, recognizes the type of the gestures through multi-layer decision-making and then sends the result to the driving module. Finally, driving module brings the correct response by using certain judge rules on the contrast result between current image and the former image, and generates corresponding message that can simulate the operation of mouse in the operation system.In the experiment we captured a short video stream containing about 600 gesture images of various kinds. The result on whole process satisfies the requirements of human-computer interaction and the expected goals: recognition ratio of various gesture type ranges from 73 percents to 99 percents, the correctness ratio of system's interaction reaches 100%, and the delay of response is less than 0.5 second.