Research On Virtual Sand Painting With Gesture Recognition Based On Leap Motion

With the rapid development of virtual reality technology and the continuous promotion of somatosensory devices,people tend to use natural and convenient way to achieve human-computer interaction,among which gesture recognition is the most convenient one.Thus,the research and development of gesture recognition in virtual reality system is of great significance to the human-computer interaction.Sand painting art has just emerged in the early 21 st century,which is an artistic creation combining sands and animation.Because it brings new audio-visual experience,sand painting art is sought after by hundreds of people.In recent years,virtual reality technology gradually integrated into skills of the sand painting art,forming virtual sand painting.Therefore,taking the virtual sand painting as the application background,the theory of gesture recognition algorithm and the key technology of Leap Motion were systematically studied in this paper,a new gesture recognition algorithm was innovated,and eventually the virtual sand painting system based on Leap Motion and Unity 3D was designed and realized.The main details of the work are as follows:(1)The algorithms of gesture recognition have been thoroughly researched.With the combination of Sparse Representation and Compressed Sensing,an algorithm of Sparse Representation Classification(SRC)was proposed.Based on SRC,the K-Nearest Neighbor(K-NN)algorithm was introduced into SRC,so the modified algorithm was proposed which was called the KNN-SRC algorithm.Its feasibility was verified from two aspects in both theory and experiment.(2)Combined with KNN-SRC gesture recognition algorithm,a gesture recognition model based on Leap Motion was designed and implemented.Making full use of the accuracy characteristics of Leap Motion when capturing gesture information,data acquisition,feature vector extraction,fingertip position detection and tracking were completed.At the same time,the feature extraction algorithm was optimized,and the tracking algorithm was improved.Finally,the effectiveness of the improved algorithm was tested through a series of experiments,which lain the foundation for realizing the virtual sand painting system.(3)The gesture recognition model based on Leap Motion and the related improved algorithms were applied to virtual sand painting system on Unity 3D platform.Gesture semantics was defined by writing the script file,the sand painting was conducted after realizing gesture recognition,and the sand falling effect was simulated by the particle system.The virtual hand model built by 3D Max was imported into Unity 3D to enhance the realism of the system.