Research On Gesture Interaction Based On Leap Motion In Virtual Scenario

With the progress of society and development of science and technology,people's demands for every aspect continue to increase.Whether in material or spiritual level,people have more personalized demand and rational suggestions.More and more people desire and pursue a more comfortable,warm,intelligent and convenient living and working environment.Human-computer interaction has gradually become a hot research subject in the current era.As one of the most commonly used interactive means in the field of human-computer interaction,gesture interaction has important theoretical research meaning,broad application background and considerable economic value.In this thesis,by analyzing the research status of gesture interaction,we know that the existing gesture interaction are mainly about data gloves and visual information,however,the current researches on data acquisition,feature extraction,gesture recognition and virtual interaction still have some problems.For example,there may have obstacles or insufficient light in data acquisition.There also have some problems in feature extraction and gesture recognition,such as the omission and deficiency of gesture information,low accuracy of algorithm and so on.In order to solve these problems,we make research on data acquisition,feature extraction,gesture recognition and virtual interaction in this thesis.The main research works are as follows:1.The research on gesture interaction system framework based on Leap Motion device and the method of building our own dataset using Leap Motion:Connecting the four modules of data acquisition,feature extraction,gesture recognition and virtual interaction to get a complete gesture interaction system.At the same time,five gestures are collected by Leap Motion,and the gesture dataset needed in the thesis is obtained.Compared with traditional dataset,the dataset has more extensive applicability.2.The research on an improved dual-finger angle domain feature extraction algorithm:The direction and three-dimensional coordinate information of palm and fingertips are obtained by Leap Motion,the projective distance and angle between adjacent fingertips are calculated based on 3D information,and it is classified by k-nearest neighbor classifier(KNN).The simulation results show that the improved feature has more objective classification effect and higher accuracy than single-finser feature based on Euclidean distance or Manhattan distance of KNN classifier.3.The research on an improved gesture recognition and classification algorithm:a k-nearest neighbor classifier based on entropy-weight allocation(WE-KNN)is proposed.By adding weights to the traditional KNN classifier,the proposed algorithm redistributes features with different proportions.The experiment results show that WE-KNN has more ideal recognition accuracy than KNN.On this basis,a double classification algorithm based on support vector machine(SVM)and WEK-KNN is researched,which is called SVM-WE-KNN.The simulation results show that SVM-WE-KNN has better separability than KNN and WE-KNN.4.The research on a virtual interactive application gym scenario based on Unity 3D:3D Max is used to object modeling and Leap Motion connects with Unity 3D,and the sgym scenario which is drawn by 3D Max is imported into Unity 3D.And the natural interaction between different gestures and virtual scenario is realized.The research is based on Leap Motion dataset.,including five gestures:"scissors","stone","paper","ok"and"good".The simulation results show that the improved feature extraction algorithm saves more complete information and improve accuracy.What's more,WE-KNN enhances separability of the dataset in gesture classification,SVM-WE-KNN can complete the second decision of classification result and further improve gesture recognition rate.These improved algorithm are applied to virtual gym scenario,which improves users'experience,enhances immersion and interactive fun.