The Research On Observation Likelihood Model For3D Hand Tracking

Human-Computer Interaction(HCI), as a branch of computer science, has gained growingattention. As the increment of mobile computing devices and the enhancement of theperformance of computer, traditional HCI style is becoming more and more inconvenient andgreatly limits the way of people’s expression, the research to develop a natural and harmoniousHCI system is becoming urgently and necessarily. As one of the people’s important expressingtools, human hand can naturally and conveniently expresses human’s interactive intention. Theexploration to gesture interaction is benefit to promote the development of harmonious HCIinterface. At the same time, gesture interaction synthesizes the knowledge of many fields, suchas computer vision, artificial intelligent, image analysis and parallel computing, its growth willalso prompt the development of these fields.The main task of vision-based3D hand tracking is to recover the hand state accordingcaptured frame image. Hand has a complicated articulated structure,26degrees of freedom areneeded at least to describe its movement, so the hand state is always represented by a highdimensional vector. In the monocular condition, recovering high dimensional hand state fromcaptured2D frame image faces two problems. The first problem is that the asymmetry between2D observation space and high dimensional state space causes their matching is divergent, thatis multiple states have the same observation. The second one is that searching the optimal statein such a high dimensional space will brings a huge computing cost, lower the system’sreal-time. So using vision information to implement3D hand tracking is actually a challengingissue.This paper is supported by National Natural Science Foundation of China (No.61173079and No.60973093), and Key Project of Natural Science Foundation of Shandong Province(ZR2011FZ003). This research refers to several key issues in the monocular3D hand tracking.In3D hand tracking, observation likelihood model is used to give a similar probability to thehand state according current observation. The similarity between observation and hand state ismeasured by comparing observation feature with hand model feature. The research of this paperis around the observation likelihood model, exploring the extraction of observation feature,how to build the likelihood model according the extracted features, how to lower the tracking degrees of freedom, and how to lower the needed sample number. The main work of this paperare as follows:(1)Hand extraction based on skin-color featureExtract hand region from frame image using two methods based on skin-color feature. Inthe first method the distribution of skin-color is modeled by using Gaussian Mixed model inYCrCb color space. After building the Gaussian Mixed model, a threshold is set to classifyskin-color pixel. The training process is implemented in an online style. In the initializationstage of hand state, sample skin-color in the hand region in real-time, then quickly obtain theGaussian Mixed model’s parameters from the skin-color samples. This kind of training couldhave good adaptability to lighting condition while reducing the required number of samplesThe second method is implemented by combining Bayesian skin-color classifier withbackground subtraction to extract hand region from frame image. Bayesian skin-color classifieris simple and easy to implement, so it is widely used to extract skin color pixels from handimage. The classifier is built by firstly training skin color samples which are a series of handimage. The quality of these samples and the setting of threshold have a great effect on thesegment performance, so the extracted hand region may not very preciously. Backgroundsubtraction extracts foreground from current image by cutting background image. But when thehand comes into the scene, it may arise the changing of lighting, so a shading region will beproduced. Background subtraction may also treat the shading region as foreground, this can besolved by using Bayesian skin color classifier when the shading region is non-skin color. On theother hand, when the segmented result obtained by Bayesian skin-color classifier is notprecisely, the reminded pixels which belong to background could be dismissed by usingbackground subtraction. So Bayesian skin-color classifier and background subtraction iscomplementary, the result extracted by their combination is better than using either Bayesiancolor-classifier or background subtraction. In addition, the arm wearing a non-skin colorclothes will be mistaken for foreground when using background subtraction, and it can beremoved by using Bayesian skin-color classifier.(2)Using extracted hand region and image edge to build an observation likelihood modelAfter extracting hand region from frame image, a similarity measurement function can bebuilt according the overlap area between hand model projection and hand region. The extraction of hand region is easily affected by the lighting condition, if the extracted result is notclear enough it will successively affects the precision of similarity measurement function.Image edge, a feature based on gradient, is robust to lighting change. In addition, whenocclusion occurs between fingers, their edge is still available. The second similaritymeasurement function is built according the matching degree between model projectioncontour and image edge. The first similarity measurement function calculates the similarity bycounting the number of pixels of the non-overlap region. The similarity given by the secondsimilarity measurement function is measured by the Chamfer distance between image edge andthe contour of hand model projection. To effectively apply these two similarity measurementfunctions into the particle filter which is successively used to track hand movement, acombinational method is defined according the particles’ weights which are respectivelycalculated by the two similarity measurement functions.(3)Introducing hand constraints to hand model to lower the model’s degrees of freedomIntroduce the general hand constraints to the3D hand model, using the static handconstraints to limit each joint angle’s moving scope and using dynamic constraints to describethe linear relationship between joints angle when the finger moves. By using the dynamic handconstraints, the tracking degrees of freedom can be reduced from26to14. After introducing thehand constraints, the3D hand model is more likely to real hand, avoid the appearance ofunreasonable hand state, and lower the tracking degrees of freedom.(4)Embedding partitioned sampling strategy to particle filter to lower the number ofsamplesUsing particle filter technology to track hand movement is implemented by sampling a setof samples in the hand state space, the posterior distribution of hand state is approximated bythese samples. As the hand state lies in a high dimensional space, and a disadvantage of particlefilter is that the number of samples grows exponentially as the increase of the dimensions ofstate. To lower the needed samples number the partitioned sampling is introduced into particlefilter. The degrees of freedom are divided into global degrees of freedom and local degrees offreedom, firstly tracking global degrees of freedom and then the local degrees of freedom, fromcoarse to fine, finally obtain the optimal hand state.