The Research On Anthropomorphic Decision-making Of Highway Car Following Behavior Based On Imitation Learning

Remarkable progress has been made in autonomous driving technology in recent years.High-level autonomous driving technology not only improves travel experience,but also improves traffic efficiency and traffic safety.In the field of autonomous driving,behavioral decision-making is one of the research focuses.The driving experience of autonomous vehicles can be greatly improved by enabling the behavioral decision-making system to understand the behaviors and habits of different drivers,learn the driving actions of drivers in specific scenarios,and satisfy the individual demands of drivers.However,most of the main methods of autonomous driving behavior decision-making technology do not take into account the differentiated decision-making strategies for individual drivers in line with their driving habits.In view of the above situation,this study establishes a generative adversarial imitation learning algorithm(DDPG-GAIL)incorporating deep deterministic policy gradients,and verified its performance in virtual simulation environment.Specifically,the main contents of this paper are as follows:(1)Design an algorithm model(DDPG-GAIL)that integrates deep reinforcement learning and imitation learning for the decision-making problem of imitating the driving behavior of skilled drivers in the scene of highway following.The model directly learns the driving strategy embodied by the driver’s driving simulation data,and obtains an anthropomorphic driving strategy model similar to the driver’s driving strategy.In the process,a general and simple formula for replacing the reward function is constructed,avoiding the tedious process of formulating different reward functions according to different scenarios in reinforcement learning,reducing the complexity and improving the versatility.In addition,the DDPG algorithm is used as the strategy generation part of the model by turning the benefits of high speed of convergence and deterministic strategy generation to account.Meanwhile,the substitute reward function is added into the parameter updating formula to strengthen the adversarial game relationship between the generator and discriminator part of the model and accelerate the speed of the model to achieve equilibrium convergence.(2)Build a driving simulator based on Logitech-Prescan-Matlab/Simulink,construct experimental data collection scenarios,obtain decision-making behavior data of skilled drivers,create expert data sets,and conduct model training based on the simulation environment Highway-env.(3)Carry out a detailed analysis of the results.The discriminant values of expert data and generated data converge to 0.53 and 0.50 respectively,close to the target value of 0.5,and the model achieves Nash equilibrium.The success rate of the following behavior guided by the generator,that is,the target driving strategy,is 93.2%,the collision rate is 0.6%.The substitute reward value obtained from the generated data finally converged around 91.62,which was only 6.84% lower than the substitute reward value obtained from the expert data of 98.35.The results show that the model has an excellent training result and can accomplish the driving behavior decision-making task in the freeway following scenario with high quality.In addition,from the perspective of following data distribution,typical indicators such as following distance,following speed,headway and acceleration are selected for comparative analysis of following behavior characteristics to verify the model’s ability to anthropomorphic learning of expert driver’s driving data.The contrast results indicate that the model has excellent learning ability to the driving strategy reflected in the expert sample data,and can imitate the driving behavior of skilled drivers to make corresponding driving actions,so as to achieve anthropomorphic driving.