Real-time dynamic hand shape gesture controller

The main objective of this thesis is to build a real time gesture recognition system which can spot and recognize specific gestures from continuous stream of input video. We address the recognition of single handed dynamic gestures. We have considered gestures which are sequences of distinct hand poses. Gestures are classified based on their hand poses and its nature of motion. The recognition strategy uses a combination of spatial hand shape recognition using chamfer distance measure and temporal characteristics through dynamic programming. The system is fairly robust to background clutter and uses skin color for tracking.;Gestures are an important modality for human-machine communication, and robust gesture recognition can be an important component of intelligent homes and assistive environments in general. Challenging task in a robust recognition system is the amount of unique gesture classes that the system can recognize accurately. Our problem domain is two dimensional tracking and recognition with a single static camera. We also address the reliability of the system as we scale the size of gesture vocabulary.;Our system is based on supervised learning, both detection and recognition uses the existing trained models. The hand tracking framework is based on non-parametric histogram bin based approach. A coarser histogram bin containing skin and non-skin models of size 32x32x32 was built. The histogram bins were generated by using samples of skin and non-skin images. The tracker framework effectively finds the moving skin locations as it integrates both the motion and skin detection. Hand shapes are another important modality of our gesture recognition system. Hand shapes can hold important information about the meaning of a gesture, or about the intent of an action. Recognizing hand shapes can be a very challenging task, because the same hand shape may look very different in different images, depending on the view point of the camera. We use chamfer matching of edge extracted hand regions to compute the minimum chamfer matching score. Dynamic Programming technique is used align the temporal sequences of gesture.;The contributions made to the gesture spotting and recognition framework is listed as follows,;A novel approach which uses both the temporal warping path score and the spatial chamfer matching score for classification of gestures.;User-chosen vocabulary.;Real-time.;Incorporates both Hand shape and Hand motion.;Automatic Confusability Analysis.;The system runs in real time with frame rate of 15 frames per second in debug mode and 17 frames per second in release mode. The system was built in a normal hardware configuration with Microsoft Visual Studio, using OpenCV and C++. Experimental results establish the effectiveness of the system.