| With the continuous development of artificial intelligence,intelligence has gradually entered all aspects of work and life.Concepts such as smart cities,smart offices,and smart medical care continue to emerge and develop rapidly.Smart education has gradually moved from theory to campus,which has become one of the inevitable development trends of the road of intelligent education.In traditional education,teachers need to teach dozens of students or more.The traditional teaching experience:teaching in accordance with students’ aptitude and teaching in fun cannot be achieved under such a large base.Teachers do not have enough energy to give consideration to all students at the same time.Teachers can only get feedback about their teaching methods from a small number of students from occasional observations:students’expressions,action states and other listening states.Moreover,students’acceptance of what teachers teach is gradually decreasing because of that what the teacher’s teaching is often from easy to difficult.Students may not be able to keep up with the teacher’s teaching progress at a certain moment.Teachers need to observe the students’ listening status frequently in order to effectively adjust their teaching progress and methods which help to achieve better teaching results.This thesis mainly includes the following four aspects:1)In view of the problem that data in existing network education resources are redundancy and confused,and cannot be directly used for training and test action recognition model.An acquisition method of classroom teaching video data,and a semi-automatic acquisition algorithm of classroom behavior data based on object tracking are proposed in this paper.The video data collection and data processing of classroom teaching videos are introduced in detail.Crawler and object tracking algorithm are used to realize automatic crawling and processing of classroom teaching video data.Aiming at the key problem of student object extraction in classroom teaching videos,a student object tracking algorithm based on deep neural network,Kalman filter algorithm and Hungarian algorithm is proposed.The object detection algorithm based on cross-stage local network and pyramid attention network realizes the real-time detection of students’ goals in classroom teaching video,and then the object tracking algorithm is used to correlate the student object images of adjacent frames,so as to realize the student object detection and tracking in classroom teaching video.Experimental results show that the proposed method has obvious performance improvement on the constructed classroom teaching video dataset.2)In view of the characteristics of students’ classroom action,which is persistent and transient,and the problems of numerous students in classroom scenes,a students’ classroom action recognition algorithm based on deep separable convolution neural network is proposed.Firstly,the students’ object image is normalized,and then the characteristics of students’ classroom action are extracted by deep separable convolution.Finally,a classification algorithm is used to obtain the class of students’classroom action.In addition,combined with the real-time detection and tracking of classroom students object,the recognition of multi-person students’ classroom action in classroom teaching scene based on deep separable convolutional neural network is realized.The experimental results show that the proposed algorithm is suitable for the classroom teaching video dataset,and the accuracy is better than other comparison algorithms.3)Aiming at the problem that students’ classroom action not only has the characteristics of retention and transience,but also needs to learn the characteristics of time sequence,a student classroom action recognition algorithm based on spatio-temporal attention neural network is proposed.Firstly,the continuous student object image is obtained by using the real-time detection and tracking algorithm of classroom student object.Then the continuous student object images are normalized,and then the spatio-temporal separation convolution is used to learn the spatio-temporal characteristics of students’ classroom action,and the efficient transformer is used to further mine the deep temporal characteristics,so as to realize the recognition of students’ classroom action based on spatio-temporal attention neural network.The experimental results show that the proposed algorithm is not only suitable for the constructed classroom teaching video dataset,but also suitable for the public dataset,and the accuracy is better than other comparison algorithms.4)Combined with the acquisition and processing of classroom teaching video data,the detection and tracking of classroom students’objects based on deep learning,and the recognition of students’ classroom action based on deep learning,a student classroom action recognition system based on deep learning is designed and implemented.The system consists of four functional modules:classroom student object detection module,classroom student object tracking module,classroom student action recognition module and intelligence classroom evaluation module.It realizes the real-time detection function for many students’ goals in the classroom student object detection module.It realizes the function of real-time tracking of many student objects in the classroom student object tracking module.It realizes the function of identifying the classroom action of many students in the classroom action recognition module.In this paper,classroom teaching video data acquisition and processing,classroom student object detection and tracking based on deep learning,and student classroom action recognition based on deep learning are implemented.A student classroom action recognition system based on deep learning is designed and developed.Both experimental and test results show that the system realizes deep feature learning and accurate recognition of students’ classroom action.The easy-to-expandable algorithm interface and the easy-to-use interactive interface are designed and implemented,and the functions of each module of the system are fully demonstrated. |
