Research On Vehicle On-ramp Merging Control Method Based On Deep Reinforcement Learning

The merging area of is one of the traffic bottleneck areas of the expressway and urban expressway.During the merging process,the main road vehicles and the on-ramp vehicles interfere greatly with each other: the low-speed lane change of the on-ramp vehicles causes the main road vehicles to reduce their speed significantly,resulting in a "go-stop" Under the condition of dense traffic flow,vehicles on the main road refuse to accept the merging of on-ramp vehicles,resulting in the inability of on-ramp vehicles to change lanes and frequent queuing overflows.The current rule-based and optimizationbased research methods for on-ramp merging control are limited by their own capabilities and computational efficiency,and it is difficult to deal with the problem of on-ramp merging under dense traffic conditions.The Deep Reinforcement Learning(RL)method has strong ability to deal with nonlinear problems and low computational burden,and is suitable for breaking through the bottleneck problem of ramp confluence and improving the traffic conditions in the confluence area.In view of the above problems,this paper proposes a set of vehicle-ramp merging control methods based on deep reinforcement learning algorithm,and the effect of the method on traffic improvement in the on-ramp merging area is verified through experiments.The main research contents and innovations of this paper are as follows:(1)According to the service level standard of highways and urban expressways in my country,the ramp-converging road network environment is designed and built based on the SUMO simulation platform.The vehicle merging simulation analyzes the bottleneck characteristics of the ramp merging area.By loading the traffic flow in one direction,the running benchmark of the main road and on-ramp vehicles is established;by changing the arrival rate of the on-ramp vehicles,the merging simulation under different traffic density levels is realized,and the congestion situation in the merging area under different traffic conditions is simulated;Based on the simulation results,the characteristics of traffic problems and incentives are analyzed for the bottleneck area,and the vehicle control optimization objective is proposed.(2)The process of intelligent networked control of the main road and on-ramp vehicles in the on-ramp merge area was formulated respectively;based on the deep reinforcement learning Proximal Policy Optimization(PPO)algorithm,the vehicle onramp merge control decision-making model was formulated.The state space,action space and artificial neural network of the control decision model are designed,and the reward mechanism is designed based on safety,lane-changing efficiency and expected lanechanging conditions.Under the different arrival rates of vehicles on the ramp,the vehicle intelligent network control method is used to control the main road and on-ramp vehicles and the traditional artificially driven vehicles in the other direction to conduct a confluence interaction experiment,and analyze the experimental results to determine the effect of the control method proposed in this paper.Finally,a feasible vehicle-ramp merge control method is obtained.The vehicle-ramp merging control method proposed in this paper significantly improves the merging process,and reduces the degree of mutual interference between main road vehicles and ramp vehicles: the main road deceleration and "go-stop" fluctuations are significantly alleviated,and the propagation range is reduced;the speed difference of lane-changing merging Significantly reduced,and safety significantly improved;on-ramp vehicles can change lanes in time,and the success rate of merging is significantly improved.