Violation Detection Of Driving Test Based On In-Vehicle Video And Audio Information

With the rapid development of transportation technology and the automobile industry,the number of motorists is growing rapidly.A fair and impartial driving test is the basis for improv-ing the overall driving quality of the people,but there is a lack of effective supervision means for security officers,who may have verbal cues,gesture cues and other cheating behaviors to help students pass the test.In order to establish an effective supervisory platform for safety of-ficers,a driving test violation recognition algorithm based on in-vehicle monitoring video and audio is designed to detect the verbal cues and gesture cues behaviors of safety officers,and the main research contents are as follows.For verbal cues detection based on audio information,an audio classification dataset is con-structed,and a lightweight audio scene classification network,DT-Audio Net,with multi-feature fusion based on MFCC and Mel-spectrogram is proposed.Noise Adversarial Module and Noise Block are designed to substantially improve the performance of verbal cues classification.For verbal cues detection based on video information,a lip detection dataset is constructed,and a YOLOv5s-based security officer lip detection algorithm is proposed.The bottleneck of the backbone is replaced with a Ghost structure to substantially improve the inference speed of the model while guaranteeing the detection accuracy,and CA module is introduced to further enhance the feature expression capability.Finally,DT-Lip Net,a 3D lip action classification network,is designed and combined with DT-Audio Net,to achieve low false detection rate and low miss detection rate of verbal cues cheating behavior detection.For gesture cues detection based on video information,an upper limb keypoints detec-tion dataset is constructed,and a Simple Baselines safety officer keypoints detection algorithm with Shuffle Net v2 as the backbone is proposed.A one-dimensional gesture cues classification network,DT-Gesture Net,is designed to achieve the detection of gesture cues.The experimental results show that the proposed algorithm can detect the safety officer vi-olations with high accuracy,which provides a solution for the automated supervision of driving tests,regulates the behavior of safety officers,and promotes the fairness of the tests.