Research On Action Recognition For Students In The Course Based On Deep Learning And Design Of The Related Analysis System

If students are the young power for a country to thrive,and teachers are the gardeners who work hard to educate people,then the classroom is an important place for teachers to teach their studies and students to master knowledge.Through digital technology detection and analysis of different action of students in the classroom,not only can remind students to regulate their own behavior,but also reflect the level of classroom activity,and help teachers improve teaching methods.At the same time,in order to meet the requirements of rapid sharing of excellent educational resources in various regions,video recording and broadcasting technology has been developed.At present,the mainstream video recording and broadcasting system on the market is still manual oriented,which requires professional personnel to operate the camera for shooting,resulting in the instability of shooting quality and the improvement of labor cost.In addition,the photographer's behaviors such as operating the camera or moving around in the classroom may interrupt the teacher's thinking or distract the students,and affect the quality of classroom teaching.Although some of the current intelligent recording and broadcasting systems use artificial intelligence technology,but only use single technology such as face recognition or voice recognition to analyze students' classroom performance,which has poor adaptability and certain limitations in the actual complex and changeable classroom scene.In view of the above problems,this paper mainly studies the role of data fusion in improving the effect of classroom students action recognition.Among them,it systematically studies the classroom student action recognition algorithm of various data types,such as human body keypoints and RGB images,the classroom student action recognition algorithm based on data fusion method,such as the fusion of human body keypoints and RGB images,and how to further improve the effect through data postprocessing and improvement.Finally,a set of action analysis system based on data fusion for classroom environment is realized.The core work and research contents of this paper are as follows:(1)In the aspect of single data type,there is no open dataset at present,so this paper first collects and makes the dataset of classroom students action,which includes three kinds of actions: sitting,standing and raising hands,and develops a set of special keypoints based annotation tool for students action.At the same time,based on the keypoints of human body,an algorithm for effectively identifying classroom students action is proposed,aiming at the traditional method that image detection and recognition cannot ignore the problem of poor robustness caused by background interference.In this paper,the traditional classroom action recognition manual feature extraction method is transformed into automatic keypoints of human body obtained by pose estimation and action recognition based on support vector machine.Ultimately through the pose coordinates normalization(PCN),head and shoulders regional non-maximum suppression(HS-NMS)and other operations proposed in this paper,it gets further improvement.(2)In the aspect of data fusion,this paper proposes a dual stream classroom student action recognition algorithm,which integrates the keypoints of human body and RGB images.In view of the disadvantage that single data type cannot provide favorable features for recognition,this paper designs a reasonable and targeted dual flow network to integrate the features of two branches for action recognition,so as to improve the robustness of the algorithm.(3)In the aspect of data post-processing,this paper studies how to improve the effect of students action recognition algorithm more effectively,and puts forward such methods as relabeling the data with large loss according to the order of training loss value,analyzing the data according to the distribution of heat map,so as to get more suitable data for training.Experiments show that using better quality data is more conducive to model training.(4)In the aspect of system design,this paper saves the snapshot of students action based on the tracking and de-duplication method,which can reduce the false detection rate;generates the local and global action statistical chart based on students classroom position according to the action statistics results;and finally realizes the function of automatically exporting the classroom students action analysis report.According to the scheme of classroom students action recognition based on data fusion proposed in this paper,a classroom students action recognition and analysis system is designed and made.The precision and recall rate of the scheme are 92% and 96%,respectively,which are verified by a number of experiments.The scheme still has good robustness against different scenes and meets the actual application requirements.