The Research And Development Of Real-Time Vision-Based Gesture Recognition System

The computer has been found in every corner of the modern life, it has been becoming an irreplaceable tool in human’s activities in order to seek natural interaction between human and computer. The gesture is a physical action rather than verbal communication, which delivers information among the different individuals, plays an important role in our society. In order to make end-users interact in a more natural way, combining effective gesture recognition algorithm with proper hardware could be work. User interact with machine using gesture in mission operations would be more efficient, friendly, and more in line with human communication than traditional method.Development of gesture recognition system would bring more digital life experience. Due to high degree freedom, diverse, fast moving of hand, recognition gesture from video sequence is a challenge of computer vision and behavior understanding.Firstly we discuss the application and research status of gesture recognition, we focus on the development of real-time gesture recognition system and the study of vision-based gesture recognition algorithm. With introducing various definitions and classification method of gesture, Quek’s method is selected. The gesture is applied to interaction in this system which conveys the simple instructions, what’s more, it is command.We capture information by Kinect device in our system; it could capture the depth image and visible image of the current scenario at the same time. An appropriate hand tracking and segmentation algorithm is proposed for the system to use the Kinect device and development platform. Depending on depth image and the device SDK we can accurately track user and his joints in real-time. Combining with the depth image threshold segmentation methods and the skin color model of the visible image, we could extract the region of user’s hand from the image, the ROI would be prepared for the identification work in the next step.Based on the different gesture recognition modes, gesture is divided into static gesture and dynamic gesture, This thesis describes the common used feature extraction methods and classification methods in detail. Considering the efficiency and speed of Gesture recognition algorithms, we extracted histograms of oriented gradients (HOG) feature of the region of interest image to recognize static gesture, The geometry and illumination transformation invariance of HOG make it is suitable for static gesture recognition. Then the feature is processed by PCA dimensionality reduction; so we obtain static gesture characteristic vector and use it to match templates vector. Dynamic gesture consists of successive video sequence. Firstly, using quaternion representation upper body’s joint of each frame’s tracked skeleton, which is ordered by joint arrangement and connection become vector xn=(xn1,xn2,…, Xn9), as Xni is represented for the i-th user’s joint quaternion rotation. Then each frame’s vector is connected to become characteristic vector X=(x1, x2,…, xt,…, xT). The distance between the characteristic vector X=(x1, x2,…,xt,…, xT) and template vector Yn=(y1n, y2n,…,ytn,…, yTn), as n>2is calculated by the dynamic time warping algorithm, to get the recognition results and output the result.Finally, according to the system’s design requirements, we define the system to identify the specific gesture, and give the system architecture. At last, it will be realized Gesture Recognition System with Kinect device SDK, EMGU and other open source code in Visual Studio C#platform.