Research And Application Of Facial Expression Recognition Based On Deep Learning

In human daily communication,more than half of all information is conveyed by facial expressions,which reflect people's moods and states.Face expression recognition is one of the important research directions in the field of computer vision,including virtual reality,augmented reality,interactive robots and traffic safety.Face expression recognition technology is needed to improve application performance and enhance application experience.With the continuous development of facial expression recognition research,from the study of a small number of facial expressions produced in laboratory to the study of facial expressions in realistic scenes based on massive data,the method of facial expression recognition has changed from the traditional image processing method to the deep learning method,among which CNN(convolutional neural network)greatly promotes the development of facial expression recognition.Although many methods of facial expression recognition have been proposed in recent years,the existing researches are mainly based on a small number of laboratory-produced facial expression datasets,which cannot meet the needs of facial expression recognition in realistic scenes.Compared with traditional facial expression recognition technology based on shallow features or manual features,deep learning can learn higher-level feature information and has better accuracy and robustness when solving the problem of facial expression recognition in complex realistic scenes.Therefore,it is necessary to study a large number of facial expression data in realistic scenes based on deep learning.In addition,most of the research on facial expression recognition is divided into many stages,so it is very innovative to propose an end-to-end facial expression recognition algorithm.Based on the research of AffectNet,this thesis proposes a deep facial expression recognition model based on YOLOv3 object detection model and the characteristics of facial expression recognition,and designing and developing a real-time driver expression monitoring system based on the deep facial expression recognition model.The main work of this thesis is embodied in the following aspects:(1)This thesis is based on AffectNet,the largest facial expression database in realistic scenes.Image processing technology(image clipping,image random rotation,image color jitter and image Gauss noise)is used to process the data set and obtain the training data that meet the requirements.(2)Based on the research and improvement of the deep object detection model YOLOv3,this thesis proposes an end-to-end deep face expression recognition algorithm.At the same time,according to the characteristics of facial expression recognition task,the learning object is adjusted to achieve the highest recognition accuracy in AffectNet dataset,and the recognition speed can reach 33 fps.(3)Aiming at the problem that drivers' expressions will affect their driving behavior,this thesis designs and develops a real-time monitoring system of drivers' expressions based on deep learning.Based on the face expression recognition model proposed in this thesis,customized models are customized according to specific scenarios,and model transformation and compression are integrated into mobile applications through Tensorflow Lite framework.Websocket,WebRTC and other technologies are used to maintain real-time communication with the back-stage management to ensure the real-time and effectiveness of the system.This verifies the application of the proposed deep face expression recognition algorithm.