Research On3D Hand Tracking Based On Multi-Model Fusion

In recent years, the interaction between human and computer has increasingly become animportant part of human daily life, especially the researches meeting the interpersonalcommunication habits are extremely active. People want to be able to communicate with thecomputer more effectively and naturally. While human-computer interaction use keyboard,mouse and handle as the main traditional tool, which is not exactly, directly and naturallyexpressing the intent of human in many cases, so as to it can not realize the harmoniousman-machine communication. Therefore, the research of natural hand tracking based on visionis an important topic in intelligent human-computer interaction, which is also a hotspot in thefield of computer vision at home and abroad.The main purpose of natural hand tracking based on vision is to continuously capture theeach frame information of natural hand motion by a camera with the algorithms of imageprocessing and tracking, reconstruct the posture and state to truly reappear the variousmovement of the current natural hand in the computer, which lies a good foundation forrealizing the harmonious, natural, convenient HIC.The main research of this article is natural hand tracking based on the monocular vision.Because human hand is complex, non-rigid and multi-link, its postures are also diversity andcomplexity. So the tracking of natural hand based on the monocular vision must exist somechangeable factors. How to ensure the tracking has higher accuracy and better real-time, that isan extremely challenging task. The research is supported by the foundation of “NationalNatural Science Foundation of China (61173079,60973093,60873089)”and ShandongProvince Natural Science Fund Project (ZR2011FZ003).This paper uses cognitive psychology and other related disciplines theory to improve thethree-dimensional hand tracking algorithm, proposes a method on three-dimension handtracking based on multi-model fusion. The main studies are as follows:(1) The state prediction model based on cognitive experiment.To establish a good prediction model is the foundation of improving the tracking precision.This paper establishes the human hand model based on the virtual interaction experiments andBenShneiderman behavioral psychology theory. First, establish the virtual assembly platform based on data glove by using data glove and position tracker. Second, combined with humanbehavior understanding and description, we obtain the hand motion data in the natural state byusing data glove and position tracker. Third, according to the stage feature of human behavior,we divide the process of hand movement into three state, they are natural state, motion state andinteraction state. We analysis the hand data according to the three state and obtain the hand stateprediction model based on the cognitive experiment.(2) The prediction model based on the local analysis.In order to get more accurate tracking results, the paper process the data of hand state inthe hand tracking process. Firstly, take n frames data of hand tracking results and use thequadratic polynomial to fit the data. Secondly, using the Sigma point distribution for the handtracking data to get the next frame of hand state, thus obtain the hand state prediction based onlocal analysis. The method uses Sigma point principle to reduce the matching error between thereconstruction hand of the virtual scene in the hand tracking process and natural hand in the realworld to improve the hand tracking precision.(3) Three-dimension hand tracking algorithm based on multi-model fusion.The tracking algorithm is proposed in this paper based on the state prediction model,which is established based on cognitive experiment and the local analysis. The two modelsrespectively match with the hand information in real world (i.e. the image information of thecurrent frame). We integrate the two models using weighting fusion according to the matchdegree, which adjust the combination of the two models each frame by different weights, inorder to restrain and guide prediction model of hand motion in the particle filter process andavoid the blindness of the prediction model, so as to obtain more accurate information of handtracking.The paper establishes the hand state prediction model based on cognitive experiments andSigma point principle to realize the state prediction of particle filter algorithm. Further, ouralgorithm compares with other algorithms in the tracking accuracy and time consumptionthrough the virtual assembly platform based on the camera, which versifies the effectivenessand superiority of our algorithm.