| Computer vision is a technology that uses hardware and algorithms to help computers to read messages from pictures or videos.Computer vision includes a variety of techniques such as image classification,object detection,target tracking,semantic segmentation,and instance segmentation.Among these techniques,image classification,object detection have been relatively mature,and are widely.In the territory of education,the number of students is large and teachers are relatively small.As a result,teachers cannot grasp the students' state of class in real time completely.Teachers and parents cannot help students more specifically based on their states.Solving this problem can effectively improve the efficiency of students' lectures and help them develop good study habits.In response to this problem,this thesis proposes a deep learning-based student sign-in and class state detection system,which uses four deep learning image classification and object detection algorithms to detect the state of the student's class,including face detection,face recognition,expression detection and fatigue detection.The results of the detection and recognition are stored and intuitively displayed on the host computer.With a platform which has high computation ability,the system's ability to process and analyze video can meet real-time requirements.Firstly,this thesis divide the system into a data acquisition subsystem,a detection subsystem and a storage and visualization subsystem,and the design requirements of each subsystem are proposed according to its functions.The main challenge of the poor expression on face recognition of Asia and expression recognition is also proposed.Then the system software and hardware design plan is given,and hardware platform is built.After that,this thesis introduces the specific design method of the system software.In this section,the thesis introduces the construction of data path is introduced in data acquisition subsystem.Then the input image is transcoded in the detection subsystem,and the data format standards within and between subsystems are designed.Then the thesis introduces the structure and principle of the face detection,face recognition,expression recognition and fatigue detection algorithms used.In response to the low accuracy of face detection in high-definition pictures,several parameters have been modified to greatly improve the accuracy of the network;for the problem of low accuracy of Asian face recognition,the Asian face database has been used for fine-tuning the network;for the low detection rate of expression recognition algorithm,the face detection algorithm has been modified to improve the accuracy;this is also happened to fatigue detection algorithm,and the similar measure is taken.The accuracy rate of the face detection algorithm reached 97.52%,the face recognition algorithm is 92.20% and the expression recognition algorithm is 61.30%.The speed of some algorithms is improved,and the total parameters of the subsystem are reduced.Finally,the software design for student sign-in and class state determination in the storage and visualization subsystem is given.The software has a convenient user interface.The test and verification results show that the system can well meet the design requirements of image acquisition,detection and identification,result judgment,storage,and host computer display.The system can process 5 pictures per second which can process 1 picture per second on embedded edges computing platform The system can achieve the real-time requirement which is set by the subject. |
