| Level 3 autonomous vehicles allow drivers to be mentally and physically disconnected from driving tasks for extended periods while retaining the ability to take over control when necessary.In this process,the involvement of non-driving related tasks(NDRTs)decouples the driver from the driving-related operations,significantly affecting their takeover performance and driving safety.To address the problem of NDRTs recognition in autonomous driving failure scenarios,this thesis aims to balance recognition accuracy and model lightweightness,and applies knowledge distillation technology to build a lightweight NDRTs classification model and NDRTs detection network.Different NDRTs are explored for their impact on driver takeover,and the NDRTs takeover performance level is comprehensively evaluated.The main work contents are as follows:(1)A visual dataset of NDRTs is constructed.The driving tendencies and higher-risk NDRTs categories in higher-level autonomous driving are analyzed,typical NDRTs based on driving distraction are selected as research objects,and a load application method is designed.The NDRTs public dataset based on driving distraction is divided into training and testing sets according to drivers,and a model evaluation dataset considering driver differences is obtained.Video data of Chinese drivers engaged in NDRTs are collected in real vehicles,and image frames are extracted at fixed intervals to produce the YH nondriving related task(YH-NDRT)dataset suitable for domestic applications.(2)A lightweight NDRTs classification model is built jointly with the knowledge relation contrastive of integrated Attention(KRCA)model compression method.Based on the characteristics of high granularity and variable spatial features of NDRTs actions,as well as the small size of the dataset,classical convolutional neural networks of different scales are used to build an Efficient Net-B0/Ghost Net teacher-student distillation architecture.To better extract the value information of the teacher model,the KRCA distillation method is proposed by combining relation contrastive distillation(RCD)loss considering sample relationships,feature attention loss based on spatial attention,and logits imitation learning,and the imitation learning across multiple layers of features is achieved through joint reconstruction of the teacher-student architecture.The decision basis of the model is presented in the form of class-specific saliency maps,and the high recognition ability of the lightweight NDRTs classification model is verified on public datasets and the YH-NDRT dataset.(3)An NDRTs detection method is built jointly with the multi-site decoupled distillation(MSDD)model compression method.A dataset for NDRTs detection is created by annotating drivers participating in NDRTs using a scheme that detects drivers first and then identifies actions.Based on the experimental results of combining different components of the teacher and student object detection networks,a basic teacher and student detection network with high recognition rates is built.The problem of suppressing student imitation learning observed in the localization distillation(LD)distillation experiment is solved by decoupling the distillation loss.To further enhance the student detection network’s ability to focus on the key pixels,channels,and global relationships between pixels in the teacher detection network,attention masks,object masks,and global context block(Gc Block)are introduced,and the MSDD method that combines neck distillation,localization and detection head distillation,and classification and detection head distillation is proposed.The high recognition ability of the lightweight NDRTs detection method is verified on public datasets and the YH-NDRT dataset.(4)The takeover evaluation method is studied using the joint criteria importance though intercrieria correlation(CRITIC)and fuzzy comprehensive evaluation methods.Based on the Sim EASY simulation platform,a takeover experimental scenario under urban roads was built,and the influence of different NDRTs on driver takeover performance was further analyzed by analyzing the variability and significance of different NDRTs evaluation indexes.To address the problems of strong subjectivity and interactions between evaluation factors,the receivership performance levels of different NDRTs were determined based on the fuzzy comprehensive evaluation method,using the CRITIC method to assign weights to the objectivity of the indicators and comprehensive multi-dimensional evaluation information.The results show that taking the phone has the highest risk and the drinking water has the lowest,for NDRT. |
