Research On Students' Online Learning Engagement Based On Multi-dimensional Information Fusion

With the development of artificial intelligence and multimedia information technology,online education system(E-learning)and massive open online courses(MOOC)have developed rapidly in recent years,and there are more and more online learners.In traditional teaching,teachers can observe students face to face and master their learning performance.However,in the network-learning environment,teachers can't monitor students' learning in real time,so they can't understand students'learning status and classroom involvement in front of the computer screen due to the limitation of network space.Moreover,learning engagement is an important indicator to measure the quality of students' learning process,which plays a decisive role in the final classroom learning evaluation.In the network-learning classroom,the learning state of students will always exist in the whole learning process and it can be analyzed and recognized by video images.Facial expressions can reflect students' learning emotions.Whether in traditional classroom or online learning,students' emotions in the process of classroom participation are very important and smiling face can best reflect students'positive learning emotion.Face posture can reflect the degree of students' concentration in learning.When the class concentration is high,students can always study hard on the computer screen.When the concentration is not high,students will turn left and right or bow their heads.The state of eyes can reflect the degree of students' learning fatigue.When the number of blinks is too much or the time of closing eyes is too long,learning efficiency is reduced at this time.The accuracy rate of answering questions can reflect the degree of students' acceptance of knowledge in class.When students are very engaged in learning,the better the absorption of knowledge,the higher the accuracy rate of answering questions.The above information is an important data to evaluate students' engagement in classroom learning.However,in the practical application of online teaching scene,the analysis of students'engagement in online learning is still restricted,which poses a high challenge to the real-time and accuracy of video image acquisition,video image processing and algorithm recognition.The effect of existing algorithms in practical application is often not ideal.In order to solve the above problems,this paper uses video image analysis technology and other related algorithms such as face location algorithm,expression recognition algorithm,eye state recognition algorithm and face pose estimation to collect the learning expression,eye state and face skew angle data of students in the video.The student's answer data is captured in the background of the learning platform.Finally,through the algorithm fusion analysis of these data,the final score of students' learning engagement is obtained.The practicability of the method proposed in this paper is preliminarily verified in the actual network teaching scene.The main contributions of this paper are as follows:(1)In order to solve the problem that real-time video images is difficult to obtain and handle,a convolutional neural network model is proposed to classify the facial expressions of students.On the premise that the model structure is compact and suitable for real-time system,the structure and loss function of the model are optimized,and the generalization ability of the model is improved.Combined with face location algorithm,the detection of students' learning emotion is realized.A convolutional neural network model which can classify the state of human eyes is proposed.Combined with human eye location algorithm,the open and closed state of eyes is recognized.And PERCLOS algorithm is used to detect the fatigue state of students.Using face pose estimation algorithm and Euler angle to calculate the deviation angle of students' face in front of the screen,the online student focus detection is realized.(2)In order to solve the problem that the information of single dimension is incomplete,a decision fusion based learning engagement evaluation model is proposed to calculate students'learning engagement score.The model combines four dimensions of learning emotion,learning fatigue,learning focus and course acceptance,and has four attributes of facial expression,eye state,facial posture and information of answering questions.This model describes students' learning state in the network teaching environment from multiple dimensions and can make a very comprehensive assessment of students' learning engagement.(3)In order to solve the problem that the existing methods are difficult to accurately express the degree of learning engagement,this paper puts forward the specific method of quantitative calculation of each dimension in the evaluation model of learning engagement,and studies the weight analysis method of multi-dimensional information fusion.First of all,the experimental data of students who watch online courses are captured and analyzed to determine the thresholds of four attributes:facial expression,eye state,facial posture and the information of answering questions.Secondly,four dimensions of learning emotion,learning fatigue,learning focus and course acceptance and their corresponding attribute weight values are determined.Finally,the scores of learning engagement are calculated by integrating the information of four dimensions to provide an objective basis for online classroom learning evaluation.(4)Based on the proposed learning engagement evaluation model and multi-dimensional information fusion method,this paper conducts an application case study in the network teaching scene.Firstly,the key information of students' online learning is grasped,and the key information is classified and fused according to the method in this paper.Then the final score of learning engagement is achieved.At the same time,the effectiveness of this method is verified by comparing with the expert scoring.The results show that the method proposed in this paper can objectively reflect the degree of students' learning engagement.