Studies On Hand Tracking & Gesture Recognition In Image Sequence

As a frontier orientation, biological feature recognition based on the dynamic image sequence has received much attention in the field of computer vision in recent years. It detects, recognizes and tracks the human from the image sequence. It also understands and describes the biological feature of human from the image sequence. The biological feature recognition of human can be applied widely to various areas like virtual reality, human-computer interaction and visual surveillance etc. Vision-based recognition of hand gestures is an extremely challenging interdisciplinary project due to the following three reasons: (1) hand gestures are rich in diversities, meanings, and space-time varieties; (2) human hands are complex non-rigid objects; (3) computer vision itself is an ill-posed problem.Special gesture set is usually defined according to the application before the system of hand tracking and gesture recognition is implemented. A full system of hand tracking and gesture recognition is composed of three parts: (1) detection and segmentation of hand; (2) hand tracking; (3) gesture recognition.Hand segmentation is to partition off the significant area (hand) from the sequence. It is the most critical step of the system in that its quality influences the results of hand tracking, hand feature extraction and gesture recognition. Tracking the hand in the image sequence, i.e. tracking in the two-dimension space, is to locate and track the hand which is projected to the image plane. It is the prerequisite to gesture recognition. Gesture recognition is to classify the tracks or the dots of modal reference space into a certain subset.The goal of this thesisis to study of hand tracking and gesture recognition for the human-computer interaction, and to focus on object segmentation algorithms, hand tracking algorithms and gesture recognition algorithms. Contributions of this thesis are summarized as follows:(1) A self-adaptive contour modal based on skin information is proposed. Because the concave contour can not be extracted accurately by this modal, a self-adaptive shape modal is also proposed to implement extraction of both convex and concave contours.This self-adaptive contour modal is an improved snake modal, which makes the contour lines lengthen or shorten adaptively and decreases the sensitivity to the initial contour. Thus in vision tracking the re-initialization of the contour is not necessary in the current frame. This overcomes the shortcomings of snake modal and snake's jump modal and ensures the precise extracting of the object contour. Because the concave contour can not be extracted accurately by this modal, a self-adaptive shape modal is also proposed. And the semi-contour obtained by this self-adaptive shape modal is revised, which can help to implement extraction of the convex and concave contours.(2) An anisotropic kernel mean shift tracker is proposed, which ensures the steady, valid, real-time region tracking.The algorithm proposes an anisotropic kernel, in which the shape, scale, and orientation of the kernels can be adapted to the changing object structure. The kernel is applied to mean shift algorithm to implement the object tracking, which ensures the steadiness and robustness of tracking. Based on the similarity measure of candidate modal and object modal, i.e. Bhattacaryya coefficient, the joint effects of mean shift algorithm and the adaptive contour modal can help to solve the difficulty in object tracking while the object is occluded.(3) On the basis of the above results in region tracking, static gesture recognition for human-computer interaction is implemented by using the orientation histograms of the hand contour. And on the basis of the above results in contour tracking results, dynamic gesture recognition for human-computer interaction is implemented by using HMM modal.The orientation histogram is the feature vector to represent hand gesture, which is robust to lighting changes and satisfies translational invariance and can be calculated quickly. Based on the region tracking results, static gesture recognition for human-computer interaction is implemented interface with the orientation histograms. Based on the contour tracking results, dynamic gesture recognition for human-computer interaction is implemented by taking both hand shapes and hand motion as the input into HMM modal.(4) A new exemplar-based tracker, i.e. CEE(CAMSHIFT Embedded Exemplar) tracker, is proposed to implement hand tracking and gesture recognition simultaneously in the image sequence.Traditional Exemplar-based tracker is unable to acquire the accurate extraction of hand silhouette and the accurate predication of the hand motion in complex background. Thus this thesis proposes a new exemplar-based tracker, i.e. CEE(CAMSHIFT Embedded Exemplar) tracker, which makes the best of motion information and color information of the object to implement precise tracking of hand and to aquire simultaneous gesture recognition in complex background.