Research On Learner's Gesture Recognition In The Classroom Scene

Human body posture recognition has always been a research hotspot in the field of computer vision and artificial intelligence.Learner posture recognition in classroom scenes is the application of human body posture recognition in the field of education,which has very important research significance and application value.Learner posture recognition is to distinguish the learner's performance of learning activities.Sitting,raising hands,and lowering head are the three most common postures of learners.Whether in traditional classroom teaching or distance education,the learner's body posture reflects the learner's learning state.The learner's posture recognition can effectively evaluate the learner's learning state during the learning process,which enables teachers to get more feedback information in time.It plays an important role for teachers to improve the teaching process and improve the teaching quality.Effective evaluation of learners' behavioral status has become an increasingly important research topic.In order to effectively recognize human body posture in classroom scenes,this paper proposes a learner posture recognition method based on the fusion of improved scale invariant local ternary pattern(SILTP)and local directional pattern(LDP)and a learner gesture recognition method based on Faster R-CNN.Using these two methods to process learners' postures,the results show that the proposed methods can accurately identify learners' postures in classroom scenes.The main research contents of this paper are as follows:(1)Introduce the background and significance of this research.Focuse on the research status of human body posture recognition technology at home and abroad.(2)Summarize the basic steps of human posture recognition,and introduce the relevant traditional algorithms and deep learning algorithms for each step.In the traditional feature extraction algorithms.LBP algorithm,SILTP algorithm and moment feature extraction algorithm are introduced.In the deep learning recognition algorithms,CNN,R-CNN and SPP-net algorithms are introduced.(3)In this paper,a new method for learner's posture recognition is proposed,which fuses the improved scale invariant local ternary pattern(SILTP)and the local directional pattern(LDP).Firstly,a multi-scale weighted adaptive SILTP(MWA-SILTP)algorithm is proposed.The dynamic threshold of the current neighborhood is adaptively generated according to the dispersion degree of contrast values in global and local neighborhoods,and SILTP coding is carried out to obtain adaptive SILTP.The adaptive SILTP algorithm can effectively solves the universality problem of SILTP threshold and has powerful adaptability.And the concept of multi-scale SILTP is proposed.By changing the sampling radius,the adaptive SILTPs of different scales are obtained.The adaptive SILTPs of different scales are merged with different weights to represent the image in multi-resolution.The MWA-SILTP algorithm is used to extract the feature of the learners posture image.Secondly,the LDP algorithm is used to extract the feature of the learner's posture image.Finally,the two features are merged,the fused feature has strong feature description ability.And the support vector machine is used for classification and recognition.Experimental results show that the recognition algorithm can accurately recognize learner's posture of sitting,raising hand and lowering head in classroom scenes.(4)In this paper,a learner posture recognition algorithm based on Faster R-CNN is proposed,which combines Faster R-CNN and 50-Layer residual network(ResNet-50).Firstly,ResNet-50 is used to extract the features of learner's posture image,and the features of learner's posture are automatically acquired by convolution calculation.The extracted features are universal and natural,avoiding the complicated manual feature extraction process.Then,according to the features extracted by ResNet-50.Faster R-CNN classify and recognize learners' postures pertinenty by obtaining their positions in the classroom scenes,so as to reduce the interference of complex background on learners' posture recognition.and realize the classification and recognition of multiple learners' sitting,raising hand and lowering head in classroom scenes.Experimental results show that the recognition algorithm can achieve a high recognition rate of the learner's posture in the classroom scene,and improve the learner's gesture recognition rate.