Research On Driving Behavior Recognition Method Based On Multi-modal Deep Learning

Today,traffic accident cause considerable losses to individuals and their families.Any inattention of drivers could cause a serious accident.Therefore,driving behavior should be monitored in order to reduce traffic accidents.In response to the above problems and requirements,this article has done the following tasks:Firstly,this thesis studies the driving behavior recognition method based on transfer learning,which is mainly divided into the research of convolutional neural network,image enhancement and simulation experiment design and evaluation of experimental results.In the research of convolutional neural networks,the basic principles of convolutional neural networks and the basic structure of the current popular models such as Alex Net,VGG,Xception,Inception Res Net and Efficient Net are introduced.In addition,this thesis conducts model training based on transfer learning on the State Farm image data set,and evaluates the performance of the model.Specifically,State Farm includes ten common driving behaviors such as safe driving,texting,operating the radio,and talking with passengers.In the end,the accuracy rate tested on the above five models is up to89.32%.However,similar behaviors are difficult to distinguish based on images,so audio is added to the recognition process.Secondly,this thesis studies the human voice detection based on the VAD algorithm,which mainly introduces related audio data sets,common speech processing methods,VAD algorithm,the experimental design and evaluation of experimental results.In the end,the accuracy rate on 21,424 audio is 99.98% and the single run time is 9.72 ms.Finally,this thesis proposes a driving behavior recognition method based on multi-modal deep learning,which includes multi-modal data fusion,simulation experiment design and result analysis.Firstly,the theory and method framework of multi-modal deep learning is explained.Then image and audio are combined,and human voice detection is used to divide driving behavior images into two categories: Driving＿Noise,and Driving＿Utterance.Finally,the results of the simulation experiment increased from 89.32% to 96.05%.In addition,this paper designs a driving behavior recognition system that can be used to recognize driving behavior.