Research On Video-based Algorithms For Analyzing Student Behavior In The Classroom

Classroom is an important avenue for students to acquire knowledge,and their behaviors in the classroom reflect their level of acceptance of knowledge.Therefore,recognizing classroom behaviors is of great significance to education.In traditional classrooms,teachers mainly rely on observation to understand students’ performance in class,yet limited by personal energy,it is hardly for teachers to grasp the classroom engagement of each student.With the rapid development of deep learning technology and edge computing chips,classrooms are becoming increasingly intelligent,by combining target detection algorithms and human pose estimation detection algorithms to recognize student behaviors,this is conducive to teachers gaining a comprehensive grasp of classroom dynamics and improving the quality of classroom instruction.This thesis compares different algorithms for human pose estimation and select a top-down approach for detecting human poses.It focus is on recognizing four specific behaviors commonly exhibited by students: raising hands,sleeping,using a mobile phone,and writing,the main contents are arranged as follows:Initially,an improved SSD deep learning object detection model is used to detect student body in the video.Given the complexity and inefficiency of SSD model,a lightweight SSD model,SSD-M3-SFP is proposed,which replaces VGG network in origin SSD model by SFP-pruned Mobile Net V3 network.The improved model size and computation are greatly reduced than the SSD model,and the detection speed is improved..Secondly,after body detection,every student is scanned for pose estimation..To address the issue of high floating point operations in the HRNet human body pose estimation algorithm,Involution is proposed to be introduced.The improved HRNet-In has reduced the size and floating point operations than the HRNet model,while attain a higher detection accuracy at the same time.Thirdly,based on the detection of multiple human body poses,the four classroom behaviors of raising hands,sleeping,using a mobile phone,and writing are identified.The actions of sleeping and raising hands are classified using the direct coordinates of the human skeleton key-points.Additionally,the actions of using a mobile phone and writing are classified using a Vision Transformer that integrates the human skeleton features.Finally,deploy the improved behavior recognition algorithm to the embedded platform.The improved target detection algorithm SSD-M3-SFP,human pose recognition algorithm HRNet-In and behavior classification algorithm Vision Transformer are fused and converted from Pytorch model to RKNN model and deployed to the embedded platform TB-RK3588 X with NPU for model acceleration.The experimental results show that the detection inference speed can be up to 40 FPS,which can meet the real-time requirements of video detection.