Research On The Recognition Method Of Students’ Learning Behaviors In Classroom Teaching Video

Student behavior is the focus of classroom observation and analysis.Combining artificial intelligence and other technologies to achieve efficient and large-scale classroom student behavior recognition and analysis can make up for the shortcomings of traditional observation and analysis methods,enhance the feasibility of student behavior analysis,and promote the development of education informatization.However,existing research on student learning behavior recognition still has certain limitations,which are mainly reflected in classifying student behaviors in a inappropriate dimensions or fine-grainedness,and experimental data are static images or collected non-real classrooms data,and relying on single modal information to identify student behavior.In response to the above problems,this paper first summarizes the classification system of classroom student learning behaviors and analyzes the relevant visual and auditory characteristics,and then proposes a learning behavior recognition method based on visual and auditory information considering the complementarity of visual and auditory modal.This method integrates multiple intelligent technologies such as computer vision and speaker recognition.And the recognition model is trained and optimized using the constructed real classroom student behavior video dataset,which realizes the recognition of students’ learning behavior in classroom video.Finally,the prototype system of recognition of students’ learning behavior in classroom teaching videos is designed and developed,which provides an effective way to realize high-efficiency and large-scale intelligent student classroom learning behavior recognition and analysis.The main research contents of the thesis are as follows:(1)Summarized the classification of students’ classroom learning behaviors and related visual and auditory information characteristics.Based on a large amount of data observation and literature analysis,this paper summarized the classification system of students’ learning behaviors from the "speech-activity" dimension combining with the demend of realizing students’ learning behavior recognition in teaching videos by computer vision and other technologies.And then the characteristics of the audiovisual modal data in the teaching videos and the audiovisual information characteristics of the various student learning behaviors in the above classification system are analyzed and summarized.(2)Proposed a method of student learning behavior recognition based on visual and auditory information.This paper constructs a video dataset of classroom student learning behaviors using the classroom teaching video resources of the partner school,and MASTER’S THESIS proposes a method of recognizing students’ classroom learning behavior based on visual and auditory information which integrating the deep learning,behavior recognition,speaker recognition and other intelligent technologies.This method first segments the complete teaching videodata based on the conversion of classroom discourse,then obtains the speaker category information,individual student location information and visual behavior category information in each part,and finally makes comprehensive judgments of students calssroom learning behaviors based on the student learning behavior recognition rule that integrates the visual and auditory information.(3)Developed the prototype system of recognition of students’ learning behavior in classroom teaching videos.This paper consturcted the overall structure and functional modules of the system based on the needs of the students classroom learning behavior recognition.The system developed uses the method ofthe student learning behavior recognition based on visual and auditory information proposed in this article to realize student behavior recognition in teaching video,and it also support visually displays and saves of the recognition results.The system effectively assists relevant personnel to carry out classroom observation and analysis,and improves the feasibility of analyzing students’ classroom learning behaviors.