Research On Autonomous Driving Behavior Decision-Making Based On Reinforcement Learning

Benefiting from the rapid development of technologies such as artificial intelligence,the driving behavior decision-making problem of autonomous vehicles has gradually become the research focus in the field of autonomous driving.Considering the complexity of autonomous vehicle decision-making in the complex traffic environment,this paper was supported by National Key Research and Development Program of China(Grant No.2018YFB1600600),to study the longitudinal decision-making method and lane-changing decision-making method of autonomous vehicles.The main research contents are as follows:(1)Aiming at the problem that the existing decision-making methods rarely consider the diverse needs of different drivers,resulting in a low level of anthropomorphism and affecting the application experience,the driving style is studied and applied to the subsequent construction of the anthropomorphic model and the setting of the surrounding environment vehicles.Driving style is often reflected by a continuous time series,and it is difficult for traditional methods to simultaneously extract deep hidden features of driving style from time and feature dimensions,which will affect subsequent clustering results.Therefore,an improved autoencoder model combined with gated recurrent units is designed,and the preprocessed data is extracted and dimensionally reduced using the model.The K-means clustering algorithm is used to divide the driving style into conservative,conventional,and aggressive,and design a comparative experiment of different feature extraction methods based on the K-means algorithm.The results show that better clustering results can be obtained by using the combination of improved autoencoder and K-means clustering algorithm.(2)Aiming at the problems of slow convergence speed and insufficient anthropomorphic degree of the existing longitudinal decision-making model,a vehicle longitudinal decision model based on the DDPGP-GAN algorithm improved by generative adversarial network(GAN)and preferred experience replay(PER)methods is proposed.The training process of the model is divided into two stages: imitation learning and deep reinforcement learning,using DDPGP and GAN to guide the learning direction of the agent at the same time,using the output of the discriminator as an auxiliary guiding signal,designing the experiment to verify the performance of the model.Finally,the anthropomorphic longitudinal decision-making model combined with driving style is established based on the DDPGP-GAN algorithm.The results show that: the learning efficiency and reward value of the DDPGP-GAN algorithm are higher than that of the single DDPG algorithm,and its performance in terms of safety and comfort is also better,and it is less affected by the fluctuation of the state of the preceding vehicle.The anthropomorphic vehicle longitudinal decision-making models with different driving styles have certain differences.(3)Aiming at the problem that the existing lane-changing decision-making model has insufficient security and relies on external environment rewards,a lane-changing decision model based on the intrinsic curiosity model(ICM)and the improved TD3-ICMS algorithm of the safety constraint module is proposed.The TD3-ICMS lane-changing decision-making model,using the intrinsic curiosity module to speed up model convergence,improve the ability of agents to explore the environment,and use the security constraint module to improve the security of the model.Finally,simulation software was used to design simulation experiments based on different driving styles of surrounding vehicles to verify the model performance.The results show that: the TD3-ICMS algorithm can obtain higher reward value and can always maintain a high decision-making success rate in different test scenarios,which has better adaptability.