Research On Vehicle Autonomous Following Decision-Making Via Deep Reinforcement Learning

With the rapid growth of urban traffic scale and the standing increment of vehicles,car-following has become the most common driving behavior in daily driving,and it has been widely used in microscopic traffic simulation,autonomous driving.For autonomous vehicles,safe and comfortable driving will improve passengers’ satisfaction and trust,reduce fuel con-sumption and bring economic benefits for owners.This paper from the two aspects of deep reinforcement learning algorithm and car-following behavior characteristics,in order to achieve the goal of safer and more comfortable,more efficient autonomous following.Firstly,based on the naturalistic driving data,statistical theory is used to analyze the characteristics of car following behavior.Taking the I-80 data in the NGSIM data set as the research object,the original vehicle trajectory data was reconstructed by the symmetric exponential mean filtering method.Based on statistical theory,the frequency distribution of each characteristic parameter in the process of following the car is analyzed in detail,and the Spearman correlation coefficient method is used to quantitatively analyze the correlation between each characteristic parameter,and its significance is clarified.Secondly,an autonomous car-following decision-making model based on deep reinforcement learning is established by using the driving characteristics.Based on the reinforcement learning theory of MDP and the characteristics of vehicle longitudinal kinematics,the state space,action space,multi-objective reward function and termination conditions are designed,focusing on the exploration and utilization of action and multi-objective reward function analysis.First,the speed-acceleration distribution is established using the training set,and the variable constraint are implemented according to the normal distribution of 3σ boundary.Second,the multi-objective reward function is designed by referring to the characteristics of car-following behavior combined with the safety,efficiency and comfort driving objectives,in which the novel driving efficiency reward function is designed according to the THW probability density distribution curve.At the same time,the agent absorbs the bad experience and introduces the penalty term in the form of mechanical energy.Third,the DDPG algorithm with the ability to explore complex environment is adopted,and the agent-environment interaction model of vehicle autonomous following decision is constructed by combining DDPG algorithm with I-80 data.For the emergency collision avoidance of the autonomous following process,a typical scene is established based on the data of collision accident types,and the vehicle autonomous collision avoidance decision model is established by analyzing the commonness of typical dangerous scenes.Finally,an experimental scheme is designed to verify the effectiveness and accuracy of the vehicle autonomous following decision strategy based on deep reinforcement learning.For the vehicle autonomous following decision model,five strategies are designed to evaluate the learning ability of the vehicle autonomous following decision model with the test set as the data object,and then the decision model is verified from the safe,efficient and comfortable driving level.Extensive simulations experiments validate the effectiveness and accuracy of our proposal.For the vehicle autonomous collision avoidance decision model,the simulation experiment is designed using the standard scene of C-NCAP.The experimental results show that the braking deceleration based on DDPG algorithm is smoother,which meets the requirements of safety and comfort.