Combination Gesture Recognition And Application Based On Kinect

At present, most of researches on gesture recognition are based on detection of fingertip point and palm point. If fingertip﹑palm﹑wrist and arm are detected together, combination of them is designed for gesture recognition, which can not only make the diversification of the types of gestures, but also is widely used in the field of human-computer interaction.In this paper, rough hand region is segmented by depth image and color image provided by Kinect. Precise hand region is detected from rough hand region by Bayesian skin color model. And then define a square as pixel classifier, finger and fingertip region is segmented by this pixel classifier. Ellipse fitting is performed on finger and fingertip region. The major axis of the ellipse has two endpoints, the endpoint which is furthest from the palm point is fingertip point. Finally, the contour was detected from precise hand region. The convex hull was extracted from this contour. A side point of wrist was detected. This side point of wrist as a vertex, and any point on the contour as another vertex, a rectangle whose diagonal is the line segment connecting this two points and which is parallel to the coordinate axes is drawn. Select the inscribed rectangular whose diagonal is the shortest in all inscribed rectangles and the center of this inscribed rectangular is the wrist point. Inscribed circles of contour are obtained and select the inscribed circle whose center is the furthest point from palm point and wrist point in all centers of these inscribed circles. The center of this inscribed circle is arm point. The above mentioned is also two main contributions of this paper.In the application, according to the fingertip point and palm point tracked by Open NI, index finger pointing gesture was designed and this gesture was applied to the virtual roaming system. It replaces the traditional interactive tools(mouse and keyboard) to interact with the virtual scene and controls the first perspective in the virtual scene through gesture. Gestures of the arm up and down was designed by combination of the palm,wrist and arm points and was applied to the augmented reality, which made that users interacted with virtual characters as in Wonderland. According to this three points, wrist rotation gesture was designed and was applied to simulate the bottle opening gesture in augmented reality.