Research On 3D Hand Tracking Based On Interactive Analysis

Research on 3D hand tracking is the fundamental and key project of natural hand-based interface which is a hot research area in Human Computer Interaction (HCI) in recent years. So, it is most significant to conduct thorough the research on this subject for boosting the theories and applications of HCI and promoting the application and researches of relative disciplines, such as artificial intelligence(AI), computer vision, HCI, and VR.The main purpose of 3D hand tracking based on interactive analysis is to obtain the gestures and their positions of the frame images by camera continuously with the guidance of interactive analysis and tracking theory, to reconstruct the corresponding 3D hand to achieve the various actions of real hand for natural and convenient HCI in future. Under the circumstance of monocular camer which this issue focuses on, the 3D hand gestures at timet can be obtained by prediction and filtering according to the 3D hand gestures at time t -1and the gestures frame images at timet .Compared with the 2D hand tracking technology, there are more technical difficulties in 3D hand tracking. Because a hand gesture, on itself, is multi-joint non-rigid object and characterized by multiformity in appearance. The hand gesture is characteristics of high degree of freedom (DOF), generally up to 27. Besides, it's difficult to get the gestures information during the movement of hand because of the status of the palm and fingers continuously changing. Such a high dimension of feature vectors directly led to the huge amount of computation, resulting in time cost overhead, which is the bottleneck of interaction tracking in real-time. Also, how to get parameters of 3D hand model from the image is a challenging issue.This project is motivated by, and fused into the research on natural hand particle filter tracking for real-time and robustness"National Natural Science Foundation of China (No.60973093, No. 60873089)"and the research on the method of natural hand tracking in interface of virtual prototype"Key Project of Natural Science Foundation of Shandong Province (2006G03)". The paper researches on some of the key problems in 3D hand tracking and focuses on the interactive analysis of hand movements to establish the corresponding interactive models to guide the tracking, in order to improve the accuracy and real-time. The following are innovative contributions of this paper:1) Flexible virtual 3D hand modelsFirstly, using cascade local coordinate systems and two direction vectors of hand, a new 3D hand model with a better design of thumb is developed. The presentation of articulated object is discussed in detail. Secondly, we demonstrate a method which synthetically utilizes the data glove and position tracker. According to the analysis of the rotating hand, we established the relationship between angles of position tracker and model parameters. Finally, the 3D Hand-based virtual assembly platform using VC++, OpenGL and 3DMAX is presented for the data collection during the movement.2) Research on gesture features detection aiming at improving accuracy and time cost.In order to provide better results of observational characteristics for tracking, a novel features detection algorithm under complex background is put forward in this paper. Firstly, gesture is segmented to obtain the contour by hand segmentation method based on the proposed color luminance. Secondly, according to different hand type, filter the contour points to obtain candidate points using improved curvature algorithm. Finally, determine the accurate feature positions by means of analyzing the distribution of the candidate points in the vicinity. And then, a new separation algorithm of feature points is proposed.3) The method of 3D hand tracking based on interactive analysis.The proposed tracking method based on the understanding and analysis of gestures interactions in virtual assembly platform, and then, we divide the entire interaction into the initialization phase, operation phase and selection phase to deal with respectively. Firstly, in the initialization phase, a new automatic initialization method for parameters of 3D gesture model is put forward, by which the operator and the computer can continuously and mutually adjust themselves to obtain the best initial parameters of 3D gestures; In the latter two phases, we establish the corresponding interaction models to guide the tracking algorithm as its constraints. In operation phase, the library of interactive gesture models is established by the Gaussian Process Dynamic Models (GPDMs) method; in the selection phase, by the analysis and comparisons of several ways of selected action, the best option of action is decided, and then, establish the library of interactive gesture models which are optimized in their trajectory, time cost, pleasure and feedback. These two interactive gestures models are used as the guidance and constraints for their sampling of particle filter, which can avoid blindly sampling in high dimensional space, and improve the precision and real-time of hand motion tracking in assembly process ultimately.In this paper, a novel perspective method for 3D hand tracking, analysis of interaction, is proposed, and our experimental results demonstrate that, compared with other methods, our method is feasible in improving accuracy reducing time-cost in our virtual assembly platform.