Gesture Behavior Analysis And System Realization Based On Leap Motion

Human computer interaction is the process of communication between human and computer in some way.With the development of the times,there are many changes happened for the form of human-computer interaction,and the interaction form has gradually shifted from the traditional mouse,keyboard and other interactive modes to widely used natural human-computer interaction now,such as touch control,voice control,body-sensing manipulation,etc..For the advantages of nature,intuition and easy to learn,gesture has become an important form of body-sensing control.According to the characteristic of motion,gestures can be divided into static and dynamic gestures.Body-sensing capture devices are the foundation of gesture capture,and Leap Motion is a new type of body-sensing equipment released on February 2013,which focuses on the desktop operation of the near distance,and can effectively and accurately track the movement of hands,and get the bones information and other data.Based on Leap Motion,we study both the static gesture recognition and dynamic gesture recognition,and design the corresponding application system.Firstly,the research background and the research situation of gesture behavior are introduced,and then the related algorithms of gesture analysis are analyzed in detail,which lays the foundation for the next work.Next,this paper introduces the static gesture recognition algorithm based on Leap Motion that it obtains the bones coordinates data and image data of the static gesture and corrects the image distortion,and then according to the characteristics of the above data and the structure of the letter gestures,we extract the fusion feature of geometric structure and BoF-SURF,and finally recognize A-H with Support Vector Machine.For dynamic gesture recognition,we adopt different algorithms for different kinds of dynamic gestures.For the dynamic execution gestures,we extract the trajectory vector features and use HMM for recognition.For dynamic digital/shape gestures,we firstly adopt a mapping algorithm to obtain two-dimensional gesture trajectory and then use CNN for training and classifying.Finally,a multi-functional gesture recognition system is designed and completed.The main innovations of this paper are as follows:1)We use Leap Motion to extract gesture data and combine the bones coordinate data and image data for gesture recognition,different from the traditional data type,and based on this,we accomplish the recognition of several custom gestures,and design a gesture recognition system including three applications that are music player,handwriting exercise board,intelligent monitoring counters.2)We improve static gesture feature extraction method.Not only do we extract two kinds of geometrical features which are Finger Bending Angle and Finger Distance,but we also extract the BoF-SURF characteristics for describing the internal structure of gestures.Finally,the weighted fusion features are gained,effective for the alphabet gestures which are occlusion,adhesion,and curling.3)The algorithm of dynamic gesture recognition is improved.We adopt a certain mapping algorithm to map the 3D trajectory coordinates to the 2D window in real time and obtain the corresponding trajectory images,and combine with CNN to realize the feature extraction and gesture recognition,which can transform the traditional trajectory vector pattern into image recognition pattern and get better recognition results.