The Research Of The Recognition And Application Of Teaching Gesture Based On RealSense

With the rapid development of information technology,human-computer interaction has undergone tremendous changes,and an increasing number of people are pursuing natural human-computer interaction experiences.Gesture recognition is highly valued in the field of natural human-computer interaction for its rich information representation,flexibility and intuitiveness.The traditional gesture recognition which is based on data gloves or visual sensors is difficult to be put into use in actual scenario due to problems such as inconvenience of wearing and external interference.There is relatively little applied research which combines gesture recognition and teacher multimedia teaching and there is no set of gesture recognition system which is suitable for classroom teaching at present.In order to solve the problems above,in this article,we use Intel RealSense to acquire hand depth information and bone joint information,and then establish a set of teaching gesture database.Based on the gesture database,the data preprocessing,key frame extraction,feature selection,classification identification and other relevant issues of the gestures are studied.After that,a gesture recognition system adapted to classroom teaching is established.The main work of this article includes the following sections:First,the habit of teacher operating software tools in the classroom is fully investigated,based on this investigation,we summarize and define a set of gestures that can cover most of the teacher's operations.Based on this gesture set,a complete and suitable teaching gesture database called CCNU-RSTGD is established..Secondly,an algorithm combining equal time interval sampling and K-means clustering is proposed to extract keyframes of gesture sequences in order to reduce redundant data.On the basis of detailed analysis of gesture features,the geometry and space features of hand joint points are selected and these descriptors in total of 40 dimensions are treated as the final gesture feature set.Third,based on the analysis of various classification algorithms,a classification algorithm DT-SVM-Boosting,which integrates three kinds of classifiers,decision tree,SVM and AdaBoost,is proposed.Experiments show that the accuracy rate reaches 98.63%while using this classifier to classify and recognize the gesture sets above.Finally,based on the above research,a gesture recognition system suitable for the classroom is established,and we combine this system with the starC teaching platform for practical application testing.As a result,the operation control of the teaching software tools in starC using gestures is realized.This article provides a set of universal gestures for classroom teaching,and conducts in-depth research on the process of preprocessing,feature extraction and classification recognition of the teaching gestures.In the end,the gesture recognition and multi-media teaching are combined in order to make classroom teaching more flexible and lively.