Research On Technology Of Hand Tracking Based On Depth Image

Hand tracking has been introduced into human computer interaction area,which is more in line with human behavior habits.And the development of virtual reality(VR),3D animation,medical rehabilitation and other technologies also provide a wide application prospect for hand tracking.But due to many degrees,variable hand movements,and other problems such as heavy self-occlusion,robust and accurate hand tracking is still a challenging problem.This paper combines generative method with discriminative method,which utilizes depth map sequences as the input and studies the critical problems of robust and accurate hand tracking based on primary estimation of hand joints and an improved particle swarm optimization algorithm(PSO).The details are as follows:Aiming at the problem of tracking failure caused by diversity of hand movement,fast gestures and hand self-occlusion,a hand initialization scheme based on hand joints estimation is proposed.Firstly,propose an effictive algorithm of hand segmentation to remove the arm region and get accurate hand segmentation from the original depth map.Second,a simple hand model is created to represent hand movement as a hand observation model driven by a 25-degree-of-freedom(DoF)hand parameter.Subsequently,a convolutional neural network(CNN)model is trained to estimate the 3D coordinates of hand joints from depth image.Transform hand joints'3D coordinates into a 25 DoF hand parameter by coordinate mapping and inverse kinematics,and then the hand tracking process is initialized in conjunction with the transformation result and the time domain information of hand movement.The related experiment results show that the proposed scheme can effectively avoid the tracking failure caused by fast movements and self-occlusion.To solve the problems that the optimization algorithm is easy to fall into local optimum in high-dimensional space and the searching speed is slow in the process of hand tracking,we design a simple but efficient cost function,and propose an optimization scheme based on an adaptive hybrid particle swarm optimization algorithm.First,the optimization problem is put forward by designing a cost function to quantify the error between the input depth map and hand model corresponding to the hypothesis hand pose.Then,an adaptive hybrid PSO is proposed to search the optimal solution for the cost function with faster convergence and higher precision.The related experiment results show that the proposed optimization algorithm can effectively improve the convergence speed and convergence precision of the cost function.