Hand Tracking And Recognition Technology Research For Human-computer Interaction Based On Depth And Skin Feature Fusion

Human-computer interaction technology is an important part of virtual reality technology and augmented reality technology and it constructs a signal transmission bridge between human and computer systems. With the rapid development of intelligent devices, the natural human-computer interaction devices begin to grow popular, the development of human-computer interaction based on this kind of equipment has received much attention, of which hand interaction technology has been widely studied. Hand detection and tracking is the prerequisite for the natural human-computer interaction. The existing algorithms for human-computer interaction are easily affected by illumination and have poor robustness. In this paper, an adaptive hand tracking by the fusion of depth and skin color features is put forward to realize a robust tracking in complex environments.It's important to choose the appropriate feature set to describe the hand feature. The depth feature of stability and the skin color feature of the cluster are adopted to describe the hand area. The system uses the gray pixel value to describe the depth data and uses the YCb Cr space model to describe the skin color feature. Considering the deformation of hand, this paper make use of the continuity and smoothness of depth feature to propose an adaptive hand detection algorithm based on depth threshold. The algorithm uses the depth threshold to achieve different scale hand image region.At the same time the algorithm replaces small sample set with traversal to improve the efficiency of algorithm.In this paper, on the basis of the detection algorithm, an adaptive hand tracking based on fusing depth and skin features(AHT_DS) is proposed. Under the framework of particle filter, the hand tracking is converted into Bayes estimation problem. Firstly, in the process of particle predicted spread, the algorithm uses the depth threshold to achieve adaptive changes of region and obtains the hand alternative regions. Then, via skin color feature of YCbCr space, the skin color normalized histograms are established to describe the alternative regions and the particle weights are updated.Finally, the algorithm determines the hand position based on the maximum a posterior(MAP) and realizes the adaptive hand tracking. In the process of hand tracking, this algorithm monitors the variance of particle weights and uses particle resampling to solve the problem of degeneracy phenomenon. In order to verify the effectiveness of the algorithm, qualitative, quantitative and comparative experiments are carried out. Experimental results show that, the proposed algorithm can achieve an accurate and robust tracking performance in real-time with complex background.