Good interest is the starting point of creative thinking activities,learning interest affects the learning method and learning process,and guiding students to maintain a higher learning interest can effectively improve students’ learning efficiency.Therefore,measuring students’ interest in learning is an indispensable part of the educational evaluation process.At present,most of the measurement of students’ learning interest in the classroom environment uses questionnaires,case analysis methods and direct teacher observations.These methods are simple and convenient,but they often cause problems such as low level recognition rate and poor credibility of recognition results.Because the teacher’s energy is insufficient,the teaching feedback is delayed.With the support of behavioral theory and emotional computing theory,this paper proposes a three-dimensional learning interest model based on cognitive attention,facial expressions,and classroom participation,and constructs and uses a convolutional neural network to identify students’ head pose information And learning emotional information,combined with the frequency of students raising their hands in the classroom,through the method of hierarchical data fusion to obtain students’ interest in learning.The main research contents of this paper are as follows:(1)Combining behavior theory and emotion computing theory,a three-dimensional learning interest analysis method is proposed.In the classroom environment,the student’s head gesture can indicate the direction of the student’s attention,the facial expression can show the emotional state of the learning content,and the student’s behavior of raising the hand to answer the question can reflect the student’s cognition of the problem and problem solving and classroom teaching of participation.By analyzing the relationship between learning emotion,cognitive attention,classroom participation and learning interest,a classroom learning interest state evaluation system is designed from the three dimensions of student’s facial expression,head posture and hand raising frequency.(2)Identification and acquisition of students’ multimodal information in the classroom environment.Constructed and used data detection methods and analysis models based on deep learning to identify students’ learning emotion and learning attention in the classroom environment.In the recognition of emotional modal information for students,in order to solve the problems of complex training,time-consuming and poor real-time performance in the expression recognition task of ordinary convolutional neural networks,a deep separable convolutional neural network is proposed.Real-time facial expression recognition model using multi-task convolutional neural network(MTCNN)and Kernelized correlation fiters(KCF)for face detection and tracking of input images of different scales Recognition of facial expressions on the FER-2013 data set has achieved a high recognition rate of 73.8%,fine-tuning on the CK+ data set has achieved 96% accuracy,and the processing time of a single frame of face image is0.22±0.05 ms.The overall processing speed reaches 80 frames/s;in the recognition of student learning attention modal information,in order to avoid the problem of low illumination,facial occlusion and low side detection,the Res Net network is trained to estimate the student’s head posture indirectly obtain the learning attention information of the students,and use the multi-loss optimization method to increase the accuracy of the prediction.The experimental results show that the average error angles for the head pose estimation on the AFLW-64 and AFLW-112 datasets are 5.84° and 5.34°,has a high recognition rate;in the recognition of modal information of students’ classroom participation,statistically quantify the frequency of middle school students raising their hands in the question and answer activities to obtain classroom participation information.(3)Fusion of multi-modal information.Based on the three-dimensional learning interest analysis model proposed in this study,AHP and entropy weight method are used to calculate the weights of the factors influencing students’ learning interest,the weighted sum of the three dimensions of learning attention,learning emotions and classroom participation to get the students’ interest in learning.The membership function of fuzzy mathematics is used to define the interval to which the students’ interest degree belongs. |