Driver Road Rage Recognition By Combining Facial Expression And Speech

The driver road rage is now a very important factor affecting safe driving,it is due to traffic congestion in the case of driving pressure and frustration caused by the driver rage,and "road rage driver" will attack other people's car and violate traffic regulations causing traffic accidents.The research of automatic detection and early warning technology of road rage has become an important part of active safety driving technology.The driver road rage has been widely concerned in recent years,but most of the research has focused on how to avoid the occurrence of road rage from the aspects of psychology,policy and regulations,and the research on automatic detection and recognition of road rage is still relatively small.Emotional recognition research shows that human expression and speech are two important channels of emotional expression.Therefore,this paper summarizes the latest developments in facial recognition and speech emotion recognition technology at home and abroad,combined with the infrared,depth information and speech information collected by Kinect equipment,and studies the problem of driver face detection,road rage expression recognition,speech emotion recognition in driving conditions.Then,we put forward a method of recognizing the emotion of driver road rage in combination with expression and speech.Finally,we prove it by experiment.The main work is as follows:(1)We recorded Kinect driver road rage behavior database.Since these is no complete driver road rage behavior database based on Kinect,our group created the driver Infrared-D(infrared and depth)information,the driver facial expression Infrared-D information,the driver emotional speech database using Kinect.(2)We proposed a method of driver face detection based on convolution network combined with Infrared-D information.Firstly,the driver region is obtained by the fusion of infrared and depth information.Then,the convolution network detector is used to scans the image to obtain the possible position of the driver face.Thirdly,the cascade convolution network detector is used to further reduce the driver face positioning area.Finally,we use NMS(Non-maximum suppression)to accurately locate the driver face.This method is compared with a variety of existing methods to achieve better results,and the precision rate and recall rate averaged is 97.3% and 84.4%.(3)We proposed a method of recognizing driver road rage expression based on PCANet fusion face Infrared-D image.Firstly,the PCANet filter is trained using the infrared image of the driver expression and the depth image of the driver expression.And then the feature map is extracted from the facial expression of the face infrared image and the depth image by the trained filter.And then hash coding and extraction of histogram features.Finally,we adopt the extracted features to training SVM model,and recognize the driver other expression and the driver road rage expression.The validity of the driver expression recognition method is validated in the experiment and the accuracy of this method is 74.6%.(4)We proposed a method combining facial expressions and speech to recognizing driver road rage.Firstly,multitask convolution neural network is used to identify the driver speech emotion from two aspects: speech signal and speech content.Then,we need to determine whether the driver speak,if not speak,we through the driver facial expression recognition method to identify the driver road rage,if speak,we through the speech emotional recognition method to identify the driver road rage.Finally,the results of speech emotion recognition and the results of driver expression recognition are accumulated within 30 s is taken as the final driver road rage recognition result.