Acquisition Of Hand Movements Based On Depth Images

Acquisition of hand movements as an input method can provide natural and intuitive human-computer interaction(HCI). It has a wide range of applications in the field of industrial control, virtual assembly and medical rehabilitation. Compared with RGB camera, acquisition of hand movements based on depth images can get more features to distinguish the different joints of human hand and will not be affected by the light and shadow in the scene. However, acquisition of hand movements based on depth images needs to resolve the trouble of the complex hand movements and severe self-occlusion in order to predict3D positions of hand joints quickly and accurately. In this thesis, we focus on acquiring hand movements based on depth images in real-time and present a complete implementation. We use seven hand model with different size to synthesize a varied training image dataset through computer graphics method so that we can enhance the stability of different person’s hand. During the random forest training phase, the classification features consider the characteristics of the different joints of the hand so that we can accurately distinguish different joints pixels. In order to predict in real-time, we propose a weighted centroid method to calculate the positions of hand joints. We also randomly select a part of pixels in single depth image to predict the joints for high frame rate. The experiment prove that our method can predict hand joints over18frames per second on a regular computer, and also be able to accurately predict the joints’positions for three hands with different sizes.Finally, we apply the forementioned framework as an input mode to control a rehabilitation game for stroke patients and a virtual hand interactive system. These two systems demonstrate the usability and effectiveness of our methods.