Research And Implementation Of Intelligent Monitoring System For Classroom Attention Based On Multi-modal Fusion

Concentration,which characterizes the continuous and persistent listening state in the classroom,is an extremely critical part of ensuring the quality and effectiveness of education.Concentration not only affects the learning state of students,but also affects the degree of classroom completion of knowledge transfer between teachers and students.Traditional classroom quality analysis is carried out by arranging lectures in the classroom,after-class questionnaires,etc.It is highly subjective,exposing the shortcomings of classroom real-time monitoring and analysis,and cannot meet people’s requirements for high-quality classroom teaching.And look forward to.Therefore,in-depth learning is introduced into the field of student classroom monitoring,and the modern education system needs to be improved by implanting artificial intelligence to assist the modern education system,and then the overall evaluation of the classroom learning quality of all students and forming an analysis report.This can not only assist teachers to obtain the status of students’ listening in real time,promptly supervise and improve learning efficiency,but also can make personalized learning plans through analysis reports and the completion of students’ homework.At this stage,classroom quality monitoring includes wearing headbands that monitor students’ brain electricity,camera equipment in the classroom,etc.These collection devices have more or less design defects,and the dimensions are relatively single,and they cannot accurately reflect students’ Learning Status.In view of the above situation,this paper uses deep neural network to analyze the collected video images,and combines the collected posture information to perform multi-modal information fusion,and realizes the comprehensive evaluation of students’ classroom concentration.The main work of this thesis is as follows:(1)Investigated the existing monitoring methods of students’ classroom concentration at this stage,compared the parameters and implementation conditions covered by various monitoring systems,compared the advantages and disadvantages,and determined the intelligent monitoring of classroom concentration based on multi-modal fusion Feasibility of system research.(2)It is determined that the main research of the system is the real-time collection and detection of image information and posture information,and the overall analysis and design framework of the system is designed,which includes head-mounted information collection equipment,network cameras,wireless routers,wireless APs,system servers,and AI servers,NVR means network video recorder and other modules.Develop a system analysis process,and finally send the focus information to the mobile phone in a graphical and report format.Determine the functions implemented by the system as a whole,including server functions and APP functions,and follow the principles of system openness,scalability,and reliability.(3)Focus on data processing platform design,including communication module,data processing module,data analysis module,data fusion module and other functions.Among them,the communication module includes video image transmission and data transmission,and real-time video streaming.The data processing module includes the processing of video image data,posture data and text information.The data analysis module is based on a deep neural network multi-target detection method(this system uses the YOLOv3 model)to accurately detect the target of interest in the video,determine the position of the target of interest in the wearer’s field of view,and further judge the wearer’s current State of concentration.The posture data can reflect the posture and shaking state of the wearer’s head in real time,and can assist in judging the wearer’s current state of concentration.The data fusion module merges the video image data analysis result with the attitude angle information,and then outputs the concentration result and stores it.(4)The server-side program design,in order to make the monitoring and analysis results of this project have a clear presentation,a server-side program is needed to provide users with a visualized result query solution.At the same time,considering that the complexity of the initial application development should be reduced and the program should have higher convenience when users use it,a server program with a B/S(browser/server)architecture was designed.This service program is built based on Express.JS and My SQL technology.The My SQL database of this program works in the data layer of the system software block diagram,providing persistence services for the monitoring results of each run.The server side of this program is supported by the Node.js language,and it runs on the server side developed based on the Express.JS framework.The server is located in the application layer and provides visual interactive interface display and data persistence functions.Finally,this research has successfully developed an AI analysis system and APP terminal program,which can monitor the concentration of students in the classroom in real time and form an analysis report,which plays a great auxiliary role for teachers to manage the state of classroom students’ listening.In addition,this system has the advantages of multi-modal information fusion,large data expandable space,flexible and convenient information collection equipment,etc.It can be applied to a variety of scenarios and has a broad market prospect.