| Because of the convenience and strong expressive force of human-computer interaction,gestures are widely used in sign language and human-computer interaction system as a non-contact interactive mode.Gesture recognition has gradually become one of the indispensable factors in the field of human-computer interaction on account of its flexibility,naturalness and intuition.In addition,the design of human-computer interaction interface provides a more attractive way for gesture interaction.Nowadays,the society has entered the stage of informatization and intellectualization,and the extensive human-computer interaction interface has become the current research hotspot.Education is an important factor in social development.The use of gestures in teaching can not only reduce the pressure of teachers,but also enhance students' sense of immersion.Based on the interactive virtual teaching platform and the ultimate goal of “a good three-dimensional geometry course”,this paper puts forward a new method of multi-modal gesture recognition in view of different device fusion,aiming at the problems of low recognition rate,poor interactivity and low robustness caused by single gesture and recognition method in traditional gesture interaction.And establish a human-computer interaction interface based on virtual teaching.The main research contents and innovation points of this paper are summarized as follows:(1)A dynamic gesture recognition algorithm based on Hausdorff distance is proposed.At present,most of the recognition methods for dynamic gestures are relatively single,and the universality is poor.This paper using digital glove to obtain the angle change data of the finger joints during the gesture process,the dynamic gesture recognition is carried out by fitting the curve and calculating the Hausdorff distance between the gestures,and the complex gesture recognition is transformed into a simple distance calculation problem.Experimental results show that the accuracy of 10 commonly used dynamic gestures can reach 98%.The algorithm has the characteristics of small computation and high efficiency,which can guarantee the correctness and robustness of the algorithm.(2)A multi-modal gesture recognition algorithm based on multi-device fusion is proposed.At present,there are few ways to combine visual information with motion information forgesture recognition.And aiming at the existence of different identification methods for different modal gestures,we propose a unified recognition model and algorithm in this paper.By using data glove and Kinect to obtain the angle change data of the finger joints and the moving change data of the hand centroid respectively.This kind of heterogeneous data is preprocessed,and all gestures are identified as curves while solving the hand jitter problem.The Pearson correlation coefficient is improved for the first time to construct the gesture similarity operator,and a unified gesture recognition algorithm is established for gesture recognition.Through the identification test of 50 kinds of static gestures,dynamic gestures and trajectory gestures,the experimental results show that the method proposed in this paper has 97.7% recognition rate.It also can cope with complex gesture interaction.(3)The concept of micro-gesture and the new recognition algorithm for it are put forward.In order to enhance the nature and accuracy of the interaction on the basis of improving the gesture recognition rate,a new gesture recognition algorithm based on micro-gesture is proposed,which makes full use of the three-dimensional spatial information of the near surface space,constructs the micro-gesture recognition model.This paper regarded the tiny and abundant movement information of the finger joint as a definite time series,and the trend characteristics of the sequence are extracted,so that the micro-gesture recognition is carried out.Through the comparison of experimental results and analysis,it can be shown that this algorithm has a good recognition effect for the recognition of tiny motion gestures.It can distinguish micro-gestures and ordinary large gestures or noise gesture at the recognition level.At the application stage,micro-gestures and general large-scale gestures corresponding to the same semantics are mapped to the same interactive operation.This paper also makes statistics on user experience based on micro-gesture recognition.Statistics show that the interactive load of micro-motion is much less than that of large-scale gesture operation.(4)The design and application of virtual scene oriented to interactive teaching interface.In gesture recognition,static gestures,dynamic gestures,and trajectory gestures are collectively referred to as multi-modal gestures.In order to support the interaction of multi-modal gestures,the study built a large dataset with 50 gestures,including 20 static gestures,14 dynamic gestures,and 16 trajectory gestures.In addition,a series of data preprocessing works are carried out for this dataset,including smooth noise and thegeneration of gesture sequence curves.In this paper,we designs and establishes a virtual hand model which can reflect the detail characteristics of the hand,and a three-dimensional simulation classroom scene.The data glove data is passed into Unity 3D in real time through UDP as the control semantic input.The virtual hand model is matched with the real manual,so that “teachers” can interact with virtual goals and scenes. |
