Research On Learning Engagement Recognition Based On Spatiotemporal Self-attentio

In recent years,under the background of the new crown epidemic,online learning has become a research field that has attracted much attention because of its convenience and security.Learning participation is an important indicator to measure the quality of students’ online learning.Research on how to effectively and accurately measure students’ learning participation plays an important role in keeping teachers informed of students’ learning status and ensuring teaching effectiveness.It can also provide key technical support for the realization of intelligent distance education system.Therefore,this paper conducts in-depth research on the online learning participation recognition method based on the self-attention mechanism.The specific research content and work include:(1)A learning engagement recognition model based on spatio-temporal self-attention and fusing contextual information is proposed.The model consists of two parallel Time Sformer networks.The spatio-temporal features of the learner’s face and context are extracted through the self-attention mechanism,so that the model has global modeling capabilities and solves the problem of limited receptive fields of convolutional networks.The model achieved 95.4% two-category and 57.7%four-category recognition accuracy on the student participation DAi SEE dataset.The experimental results were significantly higher than the baseline of the dataset,and compared with the existing deep models,it was also competitive.(2)In order to fully mine more effective features in the video and eliminate the interference of redundant information,on the basis of the previous model,a four-stream fusion learning engagement recognition model based on spatio-temporal self-attention is proposed.The model consists of four parallel Time Sformer networks.In addition to extracting face and context features,it also uses portrait segmentation technology to subtract video background,retains student portraits for extracting body posture spatiotemporal features,and simultaneously extracts key points of human movements spatiotemporal characteristics.The model achieved 95.6% two-category and 58.5% four-category recognition accuracy on DAi SEE,further improving the recognition accuracy.(3)On the basis of the above research,a prototype system for learning participation recognition based on spatio-temporal self-attention mechanism is designed and implemented.The system is built using Py Qt5 and Baidu’s Paddle Paddle deep learning framework,and realizes the functions of analyzing and identifying participation and displaying the probability of participation level through uploading learning video clips.