Research On Online Learning State Recognition Based On Image Analysis

The continuous integration of education and information technology in the background of "Internet+education" lays a foundation for the rapid development of online education.Online education has also received more and more attention for its advantages on time and space,and the outbreak of the COVID-19 has accelerated the development of online education.Compared with traditional education,online teachers can not supervise students’ learning status in real time,therefore,there is a lack of necessary interaction and emotional communication,resulting in the failure of guaranteed on the learning effect.Therefore.the research of online learning status recognition based on image analysis provides positive social value for improving the supervision means of online teaching system and improving the teaching quality.In this thesis,an online learning state recognition method based on image analysis was proposed to study the online learning state from the aspects of face detection,face recognition.head pose estimation,and fatigue detection.The main contents are as follows:(1)A face detection algorithm based on improved MTCNN was proposed.By analyzing the structure and working principle of MTCNN,the convolution layer of the network was optimized by using depth-separable convolution to reduce the model parameters;the cross entropy loss function of face classification was replaced by the Focal Loss function to improve the model performance degradation problem caused by imbalance between positive and negative sample categories;the application scenario of online learning state detection was analyzed,the median filter was added to reduce the noise effect,and the model parameters were fine-regulated to improve the detection accuracy.(2)Face feature vectors were extracted based on ResNet model,and face recognition was achieved by Euclidean distance matching.To increase the efficiency of face recognition,a face tracking algorithm based on centroid tracking was designed.The change trajectory of the target face in continuous frames was obtained by using the principle that the centroid displacement of the same face was the smallest in adjacent frames.For the same target,only the feature description operator was extracted for the initial frame,and only the face detection was performed for the subsequent frames.The feasibility of this method was simulated and analyzed.(3)Attitude estimation of the learner’s head was achieved based on 2D and 3D face feature point extraction.The ERT algorithm was used to locate the two-dimensional feature points of the face;the depth camera was used to collect the learner’s face color map and depth map,which were aligned and fused to obtain the 3D feature points of the face;the EPNP algorithm was used to obtain the learner’s head pose;furthermore,a discrimination method of learner distraction state based on learner’s head posture recognition was proposed,and the feasibility of this method was simulated and analyzed..(4)To achieve the discrimination and early warning of learner fatigue status on account of the extraction of eye and mouth fatigue characteristic parameters.The eye and mouth state was determined by calculating the aspect ratio of the eye to mouth,followed by the extraction of the eye and mouth fatigue parameters including PERCLOS value.blink frequency and number of yawns.Finally,whether the learner was fatigued was judged by feature fusion.(5)An online learning state monitoring system was constructed.The system was composed of two parts:student-oriented learning terminal and teacher-oriented cloud.The learner status was divided into six conditions:normal,unauthorized user,out of seat,multi-user,distraction,and fatigue.The learning terminal realized the acquisition of video to be tested,the detection of abnormal state,the early warning and the uploading of abnormal state information and other functions.The cloud synchronized the abnormal state data of the learning end and displayd it.The real-time video of online learning scene was collected in the laboratory,together with the system function test and analysis carried out.The test results showed that the system was feasible.The online learning state recognition based on image analysis can effectively monitor the abnormal state of students in the learning process,so as to improve the efficiency of students’ online learning and provide necessary students’ state information support for teachers.It can make up for the defects existing in the online learning environment and promote the improvement of online teaching quality.