Dynamic Hand Gesture Recognition With Leap Motion Controller

Dynamic hand gesture recognition is a crucial but challenging task in the pattern recognition and computer vision communities.In this paper,we propose a novel feature vector which is suitable for representing dynamic hand gestures,and present a satisfactory solution to recognizing dynamic hand gestures with a Leap Motion controller(LMC)only.These have not been reported in other papers.The feature vector with depth information is computed and fed into the Hidden Conditional Neural Field(HCNF)classifier to recognize dynamic hand gestures.The systematic framework of the proposed method includes two main steps: feature extraction and classification with the HCNF classifier.The proposed feature vector that consists of single-finger features and double-finger features has two main benefits.First,single-finger features solve the problem of mislabeling which is caused by executing dynamic hand gesture in different positions.Second,double-finger features can help in distinguishing the different types of interactions between adjacent fingertips.The HCNF-based classifier considers the two main factors for dynamic hand gesture recognition: different kinds of features and complex underlying structure of dynamic hand gesture sequences.The proposed method is evaluated on two dynamic hand gesture datasets with frames acquired with a LMC.The recognition accuracy is 89.5% for the LeapMotion-Gesture3 D dataset and 95.0% for the Handicraft-Gesture dataset.Experimental results show that the proposed method is suitable for certain dynamic hand gesture recognition tasks.