Real-time Hand Detection And Tracking For Human-Computer Interaction

Hand detection and tracking is a key problem in the vision-based gesture recognition technology. This paper focuses on hand detection and tracking in the monocular video captured by a motionless camera. Not only a hand detection algorithm but also two hand tracking algorithms are presented, which have been validated on a variety of challenging sequences.This paper proposes a multi-cue hand detection algorithm. It assumes that faces in the image have been already detected or tracked. The face information is used to set the hand detection range, estimate the scanning window scale, and train the skin color model. Then skin color and motion detection are performed in parallel to extract moving skin-colored regions within the hand detection range. The cascaded classifier only scans these regions at estimated scales to search for hands. It decreases the computing time and improves the detection accuracy by combining contextual, skin-colored and motional cues.This paper proposes a novel method that integrates object tracking with segmentation in a closed loop. In the tracking module, a weight image indicating the probability of each pixel belonging to the object is computed using the Gaussian mixture models of the object and background. The EM-like iterations are performed on this image to estimate an object spatial model. In the segmentation module, a five-layer region model is established based on the object spatial model provided by the tracking module. The reformative graph cuts algorithm embedded with the five-layer region model is developed to extract the accurate object region. The five-layer region model significantly improves the segmentation result when the background has the similar color as the object.The method as described above only adopts the color feature for object tracking and segmentation. It fails to track and segment the object when the object and background is indistinguishable in the color feature space. In order to overcome this limitation, this paper proposes a second hand tracking algorithm which combines skin color, motion, and Haarlike features. Three weak trackers are built using each kind of feature and integrated in a boosted cascade. If one tracker succeeds to track the object, no other tracker needs to be carried out. Skin color and motion are able to distinguish the tracked hand from most of the background region. Haar-like features are responsible to distinguish the tracked hand from other moving skin-colored regions.