Research On Driver's Behavior Recognition Method Based On Deep Learning

The recognition of driver's behavior has important practical significance.Studies have shown that 75% of traffic accidents are caused by illegal operations.If we can identify and analyze driver's behavior in advance,and provide early warning,meanwhile regulate driving behavior,most traffic accidents could be avoided.With the development of artificial intelligence and machine learning,intelligent vehicles have emerged,and intelligent vehicles have become a popular topic of current research due to the tremendous significance.Nowadays,the demand of intelligent driving keeps increasing,and it has been commonly used in our daily life.Nevertheless,there are still many problems to be solved in the practical application of intelligent driving.For example,the perception of driver's behavior in the driving process can not be timely detected,which may lead to safety issues in the driving process.Therefore,the safety of intelligent driving needs to be considered seriously.The factors which might affect intelligent driving include drivers,intelligent vehicles,road conditions,and pedestrian conditions.In this thesis,we focus on one of the factors--drivers in intelligent driving field.For the purpose to identify the driver's behaviors more precisely,the driver's dataset can be obtained by the camera installed on the intelligent car.The computer vision technology and machine learning methods have been used to identify and analyze the data,and the driver can be reminded by his final experimental results.This thesis mainly focuses on analyzing the driver's gesture behaviors and the driver's head behaviors while driving.Along with the changes in the real application scenarios of intelligent driving,the amount of data and calculations of the datasets keeps increasing,and the corresponding application scenarios are no longer a single application scenario.Therefore,the traditional recognition algorithms can no longer satisfy the current needs of intelligent driving.In this thesis,we use the behavior recognition algorithm based on deep learning.Compared with traditional recognition algorithms,this type of method has achieved good results in behavior recognition,and the accuracy and speed of recognition have been improved.These problems include the stability of driving data,the environmental impact of data collection,etc.The existence of these problems will directly affect the accuracy of the final recognition.To better solve these problems,we target our research on the driver's gesture and the driver's head behaviors.In structure,the workflow of this thesis is:Firstly,in view of the problem of noise in the VIVA Challenge gesture dataset,a denoising method based on the combination of convolution and deconvolution is proposed.This method reduces the noise in the original dataset.Considering that the dataset contains both RGB images and depth images,a scheme of interleaving the depth images and the video frames of the RGB images frame by frame is proposed,and the residual idea is combined with the 3D Convolutional Neural Networks for training to achieve the classification and recognition of driver's gesture data.The final experimental results indicate our method have advantages in processing gesture data classification problems.At the same time,it demonstrates the efficacy of the interlaced frames.Secondly,in view of the characteristics of the Kaggle dataset containing RGB images and depth maps,the depth map and RGB image are combined to make full use of the rich information provided by the two modalities,and a multi-modal feature fusion driver gesture recognition method is proposed.In order to make better use of the effective information in the depth gesture video,we designed an improved weight-based frame selection method to select more representative frames from a given video.In addition,to use of information more effectively,features can be extracted by the Res Next network.Then the FEB method can be used for static data.Meanwhile,the information from the static data can be used.Then,the processed static and dynamic data features are integrated together through DCA(Discriminant Correlation Analysis)method to gather the information,and a linear SVM(Support Vector Machine)classifier is used for classification.The experiments indicate the accuracy of this classification method is 98.35%.Our method is better than current comparison methods,hence it can be used to recognize gestures in intelligent vehicles.Lastly,we propose a classification method based on YOLOv4 and the attention mechanism to solve the problem of the driver's head behaviour recognition.First,we use the YOLOv4 networks to train the driver's upper body dataset to extract the driver's head images in a fast and accurate way.The next step is to apply the resize technique to reconstruct all head images into the same size.Lately,in order to classify the dataset more specifically,the dataset would be reclassified to better adapt into the subsequent classification tasks.In addition,in order to extract more discriminative features,an attention mechanism(consisting of trunk and mask branches)containing global attention is proposed,and at the same time in TSE-mask(Transformed SE-mask).The experiment results indicate that this method can achieve the classification accuracy of 83.26%.Our experiment proves that this method is better than the latest method.Therefore,it can be used to recognize the driver's head behaviour.On the other hand,due to the problem of imbalance between classes in the driver's upper body dataset,it is necessary to process categories with a small amount of data,and at the same time make full use of the original images in the dataset,and use the PULSE(Photo Upsampling via Latent Space Exploration)method to restore the failed focus images can improve the problem of imbalance between classes.The self-calibrated convolution is used in the Res Ne St algorithm.The better classification effect can be obtained under the condition of the number of parameters keep almost the same.Experimental results show that compared with other comparison methods,this method can identify and judge driver behaviour faster and better.By solving the classification and identification of driver gesture behavior and driver head behavior,this paper proposes a method to ensure drivers' safety in the process of intelligent driving to a certain extent and provide more solutions for the practical application scenarios of intelligent driving.