Camera-based Gesture Interaction System

Interaction between human and computer is playing an important role in our daily life. Especially in recent, the Human Computer Interaction tools is becoming hot with the development of computer technology, that make us use the nature input method instead of mechanical method such as voice and gesture recognition is becoming focus of our research.This paper researches on hand capture technology and presents a vision-based method for finger-touch interaction which involves pattern recognition, digital image processing and recognition science. The research paper includes the followings:Firstly, in the hand segment part, this paper proposed a method of segmentation based on complicated background image. In this paper, to get better results, a multiple cues (color and motion information) fusion technology is applied to segmentation procedure. In the motion analysis, background difference is used; then color-clustering is used to localize gestures. In the segmented gesture areas, we extract skin-color object respectively in RGB-space and HSV-space. We use RGB color skin model as rough segmentation and the improvement of HSV color model as delicacy segmentation.Secondly, in the part of feature extraction part, a novel approach based on Side Traversal twice Scanning (STTS) is used for fingertip localization. We firstly get hand region of user through video tracking. Some parameters of the binary images are calculated for the centers and orientations of hand region, and then the binary images are scanned using STTS to achieve fingertip coordinates in image plane. From experiments' results, the STTS can achieve better localization precision of fingertip. Apart from this, this method has better capability of anti-jamming.Thirdly, this paper provides a vision-based system for finger-touch interaction which achieves natural interactions by finger moving and clicking on rectangular panel. The clicking actions on the panel are detected with the finger's location and its stay time. The system can be used to achieve cursor positioning of a virtual mouse.