A Research Of Deep Learning Based Students' Attention Analysis

The practical application of deep learning algorithms is an area that has attracted much attention,and classroom scenes in schools are very suitable for Deep Learning al-gorithm application scenarios,because there are a lot of surveillance video data in this scenario.Student attention analysis of video or picture data taken by surveillance cam-eras belongs to the field of Human-Object Interactions in Computer Vision.It is an ab-stract semantic understanding problem and a relatively advanced and difficult problem in computer vision..At present,there are several problems for the research results in the field of Human-Object Interactions to be directly appllied to the analysis of students' attention in the class-room:1.unlike Face Recognition or Object Detection in Computer Vision,Human-Object Interactions is a field yet to be tackled.The best performing algorithm on the representa-tive HICO dataset can not achieve 40%of mAP.2.The specific categories of attention in the field of Human-Object Interactions are relatively broad,and there is little works that focus on students' attention in classroom scenarios.3.Rarely studied algorithms or mod-els of students' attention analysis in classroom scenarios require a lot of complex labeling,making data collection a tough work.This thesis takes the analysis of student's attention in the classroom scene as the research topic.This thesis focuse on the core question:through surveillance images or videos,classifying whether the student's attention is on the blackboard.Specifically,stu-dents are roughly classified into three categories:standing,sitting down and lying tables,which are further classified into four categories:head towards the blackboard,head side-ward to blackboard,head backward to blackboard,and bowing head.The main contribu-tion of this thesis includes:1.In the rough classification of students' attention,a graph convolution network struc-ture that accepts skeletal heat map information as input is proposed,and a dual branch network is finally proposed based on this.The performance of semi-supervised learning and different loss functions on this task is also studied.2.In the detailed classification of students' attention,this thesis proposes a classifica-tion model that is robust to position-variance and perspective-variance.Meanwhile.The task was successfully transformed into a classification task.The final result using this method is significantly better than the baseline.3.In the detailed classification of students' attention,a regression loss function based on interval and mean square error loss function is proposed.This loss function can exploit unlabeled angle information in data.In experiments,it performs better than mean square error loss function.