Research On Overtaking Decision Of Autonomous Vehicles Based On Deep Reinforcement Learning

The field of autonomous driving faces a daunting challenge in furnishing autonomous vehicles with safe and effective overtaking behavior decisions due to the swift advancement of autonomous driving technology.The complexity of overtaking decision-making for autonomous vehicles,in addition to the diversity of traditional driving vehicle behaviors in the surrounding driving environment,must be taken into account.This paper was supported by National Key Research and Development Program of China(Grant No.2018YFB1600600),takes mixed traffic as the main research scenario,and conducts research on the overtaking decision-making of autonomous vehicles based on deep reinforcement learning.The main research contents are as follows:(1)Aiming at the dangerous distracted driving behavior of traditional human-driven vehicles in mixed traffic environment,the characteristics of distracted driving behavior are identified,and then applying it to surrounding driving vehicles in heterogeneous mixed traffic environments.Firstly,driving feature regions are extracted from real distracted driving image datasets to form a new augmented dataset.The newly generated dataset is identified by the optimized four-layer convolutional neural network model,and it’s recognition accuracy is then contrasted with that of the unenhanced dataset.Finally,the driving behavior of distracted,aggressive,and conservative of surrounding driving vehicles are modeled.The experimental results show that When the multi-layer convolutional neural network model is used to identify the characteristics of distracted driving behavior,the accuracy on the enhanced dataset is higher than that on the original dataset;Compared with the three-layer and five-layer convolutional neural networks,the four-layer convolutional neural network model has the highest recognition efficiency and accuracy.(2)Aiming at the problems of long training time and slow convergence speed in the existing overtaking decision-making model in mixed traffic scenarios,an overtaking decisionmaking model based on Meta deep Q network is proposed.Firstly,an improved Meta deep Q network is proposed,which obtains the initialization parameters of the deep Q network by metalearning.Then,an overtaking decision model based on the Meta deep Q network is established,and an overtaking decision module composed of a convolutional neural network is constructed in the model.Finally,a simulation environment in a mixed traffic scene is constructed,and the improved model is verified in this simulation scene.The model comparison results show that in the efficiency and safety indicators of the overtaking process,the overtaking decision model based on Meta deep Q network outperforms the overtaking decision-making model based on deep Q network algorithm,and the training time is shortened and the Rate of convergence is accelerated(3)Aiming at the problem of single task assignment of overtaking decision in existing heterogeneous mixed traffic scenarios,an overtaking decision-making model based on the MSGI-PPO algorithm improved by Meta-learner with Subtask Graph Inference is proposed.Firstly,an improved MSGI-PPO algorithm is proposed,which incorporates a subtask graph inference meta-learner to enable the proximal policy optimization algorithm to infer subtasks.Then,an overtaking decision model based on the MSGI-PPO algorithm is established,which can infers the subtasks of the overtaking tasks and quickly adapts the new tasks within several episodes in the adaptation phase.Finally,a simulation environment is built for a heterogeneous mixed traffic scene in which surrounding traditional driving vehicles have different driving styles.The model comparison results show that under the different indicators,the overtaking decision model based on MSGI-PPO algorithm performs better than Other comparison models.