3D Vision-based Hand Tracking And Its Application In The Human Computer Interaction

In recent years, human-computer interaction technology is a hot spot of computer science. As an interface of information exchange between human and computer systems, human-computer interaction technology enormously impact on human society. It is the most fundamental form of human-computer interaction that humans use the computer input device to enter command information to the computer, and then the computer system untilize the output system to show the results of its execution. Human-computer interaction technology is significant for human to use the computer systems efficiently. In recent years, rapid development of computer science greatly impoved the performance of computer systems and lead to emerging of various of sensing equipment. Accordingly, the direction of human-computer interaction technology is to make the information exchange between people and computer system in a more natural, comfortable and efficient manner. Moreover, computer systems need to be more intelligent and be able to understand the meaning of human action. Through such efforts, seamless human-computer interaction can be ultimately achieved that, in another words, to be humane and intelligent.Begin with research of human gesture, this paper invastigates the hand tracking and apply it in human computer interaction. The system is composed of a Kinect Camera, a new type of camera which can capture RGB and depth information of objects. According to the RGB and depth information collected by Kinect, this paper presents three-dimensional vision-based gesture tracking algorithm, and presents its application in human-computer interaction.The three-dimensional vision-based hand tracking algorithm presented in this paper combines advantages of skin color recognition and range image segmentation. The traditional hand detecting method through skin color recognition often relys on the restrictions of the user's location. It is difficult to identify the hand with complex background. This paper presents a new method to detect hand with complex background. Firstly, we detect the skin color in the RGB image, then segment the depth image to pick up the area around hand, and then map the depth image to the RGB image through the 3D-2D conversion calculation. Compared the results of segmentation with the results of skin color recognition, we can filter out the background interference, extract the hand image. Finally, we track the hand according to the result of hand tracking of two consecutive frames. Different to traditional detection methods, this method is flexible and adaptable for variety of user and complexity of environment, and it can track hand in the three-dimensional space.