The Research Of Hands Movement Using Depth Sensor

With the development of computer technique and sensors,the human computer interaction(HCI)has moved into the era of natural user interface interaction.As the hand plays an important role in the daily human interaction,the hand-based HCI has become fundamental to the natural user interface field.With the release of the depth sensor Kinect by Microsoft,the state of hand movement in realtime is analyzed with the technique of computer vision becomes possible.During the development of such applications,it requires to accuratly reflect the real movement of hands,which puts new challenges to the real time computation algorithm.Therefore,this thesis firstly takes the hand motion tracking as the main line,and studies the improvement of computation efficiency while gurantees the accuracy,stability and robustness of the tracked features.The main work in this thesis includes:(1)Proposing a depth sensor-based real-time hand region tracking algorithm.The forward kinematics and Kalman filter are applied to the hand tracking to improve the accuracy of hand joint from Kinect to improve the precision as it is diffcult to obtain the complete information of hand area.The recursive connected component analysis is applied to every frame to obtain the current hand region.According to the real-time requirement,the cascade structure is used to improve the efficiency.Experimental results show that the computation cost consumed in this algorithm can be reduced to 12.4% of that in the traditional solutions,with the guarantee of accuracy.(2)Proposing a real-time fingertips tracking algorithm using depth sensor.During the procedure of hand rotation and finger extension,it is diffult to rack fingtip.In this thesis,to achieve the positions of fingertips,it combines the connection information about the hand area and use the maximal minimized 3D measurement from the groud as the determination.Considering the real-time requirement,the skeleton pixels-based in hand region measurement is prosent.Experimental results show that the computation cost is about 35.9% of that in the whole hand region computation.(3)Implementing the real-time hand movement tracking using the depth sensor.To combat the problem of difficulty to achieve the specific physical meanings for the high scale feature,this thesis is able to provide features such as the hand depth region,hand position,hand skeleton,pixels,to describe the movement of hand for human computer interactions in real-time.The experimental results verify the feasibility of proposed solution.