Hand Pose Reconstruction And Recognition From Single Depth Images

Traditional input devices include keyboard, mouse and joysticks. Gestures have long been the desirable way for natural computer interaction. However, gesture recognition is a tough problem all the time. Digital gloves and marker-based method provide a way to solve this problem, but wearing extra equipment on the hand lessens the natural and general character for computer interaction.Vision-based hand gesture recognition can work without extra devices to wear on. But many of them are color-based, and such method is far from real-time usage and susceptible to background and lightWe propose a new method to quickly predict the3D keypoints of the hand using the depth data provided by Kinect and hand gesture can be described by these keypoints. We take an object recognition approach by designing an intermediate hand parts repsentation which maps the pose estimation problem into per-pixel classification of hand parts problem. After that, We use Mean shift mode-seeking approach to locate the keypoints for each hand part. For collecting enough training images, we use simple and inexpensive color glove images and depth images by Kinect, we convert each hand part into the same color by using Mean shift image segmentation operating in Lab color space.We gain the robustness by processing each frame separately without using any temporal information, preventing current frame recognition failure influence the next frame recognition. Our method can overcome the disadvantage of tranditional color-based method, such as separating the hands from background and ignoring the light influence.Our method run at25frames per second on personal computer without any optimization. It could achieve much higher efficiency by coding part of the method on GPU to run parallel with CPU.