Driver’s Facial Expression Recognition Based On Deep Learning Technology

For the last few years,since the automobile industry developed rapidly,the traffic problems have deeply troubled people’s daily life.Among them,more than 90% of traffic accidents were caused by driver’s factors like fatigue,distractions or emotions.Lately,traffic problems caused by drivers’ emotions such as "road rage" are not uncommon.Therefore,to reduce the occurrence of dangerous driving behaviors and improve road safety,accurate recognition and timely warning of drivers’ emotions are very necessary.In addition,for L3/L4 autonomous driving vehicles,drivers are required to participate in vehicle control partially,so recognition on driver emotions will help the autonomous driving system to evaluate the driver’s psychological state and take-over ability.Thus,the research on driver emotion recognition is of great significance.Based on the driver expression dataset collected in real driving scenes,this paper adopts deep learning method to fully mine the crucial information of drivers’ expressions,and proposes an end-to-end driver expression recognition framework.The main contents are as follows:(1)Construction of driver expression dataset.Considering the scarcity of existing driver expression datasets,this paper integrates facial expression datasets collected in multiple scenes for facial expression feature learning,and uses driver expression datasets in real scenarios for model evaluation.In addition,a driver expression video dataset is collected for validation of the real-time application.In view of the variable face proportion in the real driving environment,standardized face regions are cut out based on the facial landmarks,which are used as the representation of drivers’ facial dynamics.These datasets lay solid data foundations for the following studies on driver expression recognition.(2)The driver expression recognition algorithm based on transfer learning and deep learning is proposed.Because face expression task and face recognition task share the procedure of extracting facial features and the existing facial expression datasets is relatively fewer.In order to alleviate the scarcity of facial expression dataset,this paper pretrains the Inception-Resnet model based on Face Net,and then finetunes the model based on facial expression datasets,so that the facial expression features of drivers could be modeled.Subsequently,the driver expression features are input to the classification block for the corresponding result.Finally,the results on driver expression dataset KMU-FED show that the driver expression recognition model Basic-IRN can complete the task of driver expression recognition,but the overall recognition performance still can be further improved.This model will serve as the benchmark of this paper and the basis for subsequent model optimization.(3)An attention-enhanced driver expression recognition network framework with label smoothing is proposed,namely SE-IRN with LS.For the Basic-IRN network,there is still some problems: the representation of model is insufficient,the utilization of information is inadequate,and the negative expressions are still hard to tell from each other.In order to solve these problems,channel attention mechanism and label smoothing strategy are introduced.In this paper,sufficient ablation experiments,cross-validation experiments and visualizations were designed to verify the effectiveness of channel attention mechanism SE module and label smoothing respectively.The results show that the introductions of SE module and label smoothing brings different degrees of improvements.The final results tested in the video streams show that the driver facial expression recognition network can recognize drivers’ basic facial expressions from facial images effectively.