Vision Based Gestures Recognition

With the development of technologies,various smart devices,such as robots,wearable devices,smart furnishings and virtual reality glasses,are coming into our life.These products have provided great convenience for us.However,the human-machine interaction performance of them is still so poor,which has limited their application.Touch screen as a popular interaction technology has ever pushed smart phones to be successful,but it can not satisfy people anymore for its limits in volume and performance.A new,simple and efficient human-machine interaction method is urgent to be developed.Gestures are simple,natural assistant communication in our daily life,which can provide rich nonverbal information.For this convenience,gestures recognition can be a valuable technique to achieve friendly human-machine interaction.There are three common methods used in gestures recognition:data gloves based method,hand-held devices based method and vision based method.Because the vision based method uses cheap hardware and it's a natural way for non-contact interaction,this method will be more easily acceptable in most applications.The vision based gestures recognition captures images with a camera.It detects the hand and extracts features from the image,and recognizes gestures with classification algorithms.Most existing methods are sensitive to varied illumination conditions and complex environments.Thus,they still have problems such as low accuracy and poor robust.A RGBD images based gestures recognition method has been introduced in this thesis,which consists of four steps: gestures detection,gestures tracking,gestures segmentation and gestures recognition.First,a HOG descriptors and support vector machine based gesture classification is trained with depth images,which is used to detect gestures in the image.Because the hand can be highly deformable,a detector for a initial gesture is trained to get the initial position of the hand.Then,gestures tracking is applied with CAMSHIFT tracking algorithm to locate gestures in each image from images sequence.In order to get more accurate region for gestures,a multi-region-growing method has been proposed to segment the hand.Region growing and shape,depth information based weak segmentation methods are applied alternately,which can segment the object accurately.Two gestures recognition methods are introduced in this thesis.The first one is gesture shape features based template matching method,and the other one uses convolutional neural networks which can achieves end-to-end recognition.The RGBD images based gestures recognition method introduced in this thesis can achieve high accuracy and great robust.This method can even detect and recognize multiple gestures in the image at one time.At last,this method is applied to control a manipulator with one hand.It achieves that the manipulator can track the hand position and posture in real time.