Decision And Control For Automated Lane Change Based On Deep Reinforcement Learning

In order to solve traffic congestion and road safety problems,vehicles use advanced autopilot technology to replace human drivers for decision-making and control.Since the prediction results of traditional rule-based automatic lane change models for the real scene deviate from the actual results,and the model needs to be modeled according to different road scenes,this dissertation chooses the learning-based artificial intelligence algorithms to implement automatic lane change.Since the training of deep learning requires a large amount of labeled data,and deep reinforcement learning algorithms can be trained by interacting with the environment,which is more suitable for automatic lane change decision and control problems,so this dissertation divides the automatic lane change problem into a lane change decision layer and a lane change execution layer,and uses different deep reinforcement learning algorithms to learn the strategies of the two layers,while solving the problem that single-layer deep reinforcement learning is difficult to train due to parameter growth.Finally,this dissertation proposes an automatic lane change decision and control scheme with multiple driving behaviors,which can safely and comfortably implement automatic lane change decision and control in dynamic traffic environment.The main research contents are as follows:Firstly,to generate the training scenario of deep reinforcement learning algorithms,an automatic lane change scene of two lanes and four vehicles in a straight line is built in Car Sim and an intelligent driver model is established to control the surrounding vehicles to follow the driving.After that,the inverse longitudinal dynamics model of the vehicle and the switching model between the drive/brake system are built to convert the desired acceleration output from the execution layer to the vehicle control signal,and it is demonstrated by the co-simulation of Matlab/simulink and Car Sim that it can be used in the subsequent study.Secondly,the automatic lane change decision behavior is decomposed into immediate lane change and no lane change,and the deep Q-network algorithm of the decision layer and its parameters are described in detail.In order to obtain an agent with well-performing,the algorithm is firstly modeled with Markov decision process concerning input state space,output action space and reward and punishment functions;then the design of algorithm training parameters,network structure and dynamic simulation scenarios in the training scheme is developed with reference to the policy task objectives;finally the convergence analysis of the algorithm training results is carried out using the average round reward as an indicator.Then,according to different lane change intentions,the execution layer strategy is decomposed into the following of the invariant lane,the gap adjustment and the lateral control of the lane change of the lane change.Since the vehicle control signal is continuous,the twin delayed deep deterministic policy gradient algorithm is adopted,and the algorithm and unique parameters are described in detail.Also,in order to improve the utilization of the algorithm for samples,the prioritized experience replay technique is added to the algorithm Sampling step.Finally,with reference to the design and training of lane change decision strategies,Markov decision process modeling,training scheme design,and convergence analysis of algorithm training results are performed for each of the three strategies in the execution layer.Finally,the performance of the agent with automatic lane change strategy is verified.The verification order is according to the structural level of automatic lane change in this dissertation,and the specific verification method is to select one to two working conditions for simulation verification.Finally,the ability and effect of the agent to complete the strategy task under different initial conditions are analyzed.It is proved that each agent can complete the strategy task goal well,and the agent trained by the hierarchical deep reinforcement learning algorithm can realize the automatic lane change decision and control.Finally,the performance of the agent for automatic lane change strategy is verified.The verification order is according to the structural level of automatic lane change in this dissertation,and the specific validation is performed by selecting one or two working conditions for simulation validation.Finally,by analyzing the ability and effect of the agent to accomplish the strategy task under different initial conditions,it is proved that each agent can accomplish the strategy task objective,and the agent trained by hierarchical deep reinforcement learning algorithm can implement the automatic lane change decision and control.