| Human-computer interaction has greatly enriched the user experience since its birth.In the field of human-computer interaction,the tracking of human skeletons has become increasingly important,mainly due to their applications in medical,biomedicine,human-machine interface,virtual reality,robotics and other fields.As the most flexible joint in the human skeleton,the hand is considered the most convenient and effective communication tool,so the human-computer interaction based on hand movement has always been an extremely important part of the field of human-computer interaction.The advent of the depth sensor Kinect has injected fresh blood into the human-computer interaction technology based on hand movement and brought a more convenient user experience.However,in the actual application scenarios of human-computer interaction,this technology still meets challenges in terms of real-time,accuracy,and robustness.To cope with these challenges,this thesis studies the research of hand movement tracking based on Kinect in real application scenarios,mainly studying the two characteristics of hand direction and fingertip position during hand movement.The main research focuses are as follows:Aiming at the problem of poor real-time performance and low accuracy in hand direction estimation,the thesis designs a method of hand direction estimation based on Euler angles.Firstly,the Holt double exponential smoothing filter algorithm is used to smooth the bone data of Kinect,and the wrist point is located according to the bone data.Then the quaternion is converted to Euler angles representing the direction of the hand.Because Euler angles are noisy and unstable,this thesis uses a method that combines Kalman filtering and median filtering to filter Euler angles.The experimental results prove that compared with Kalman filtering,this method reduces the variance of the original data of the three Euler angles of Roll,Pitch,and Yaw to a greater degree,which are 11%,14%,and15% respectively,so that Euler angles are more stable.By estimating the direction of the hand in the color image and using Open GL to build a 3D hand model to simulate the hand movement in real time,it is verified that the hand direction estimation method based on Euler angles designed in this thesis can realize real-time and accurate hand direction estimation.Aiming at the problem of low accuracy of fingertip detection,the thesis designs a method to combine the geodesic distance with the convex hull.Firstly,whether the fingers are merged or separated is distinguished by the difference in the maximum depth value of the convex hull defect of the hand when the fingers are merged and separated,and is marked by the flag bit.Then different fingertip detection algorithms are determined according to the different flags.The experimental results prove that this method can accurately identify the position of the fingertips in all kinds of states including four-finger merging or two-finger merging,finger separation,hand bending,rotation,pening and closing,etc.,and the accuracy is high.It also has a better effect in the palm of the hand against Kinect and in a dark environment,and the robustness is strong. |
