| With the rapid development and popularization of communication technology and sensor technology,connected automated vehicles(CAVs)that can integrate modern communication and network technologies will gradually replace ordinary human-driven vehicles(HVs).CAVs has the ability to perceive the surrounding environment through the interaction with other vehicles and road facilities,so as to make intelligent decisions to achieve safe,comfortable,energy-saving,and efficient driving requirements.The current traffic has the problem of traffic oscillations caused by information lag,which generally occurs in the situation of traffic lights at the intersection.There are few researches on the scene where CAVs and HVs coexist on the road.This thesis will address this problem,using reinforcement learning technology in a mixed flow scenario,to control and optimize the trajectory of CAVs so that they can pass through the intersection in a smooth trajectory.The main contributions and innovations of this article are as follows:This paper proposes and designs a set of deep reinforcement learning framework for optimizing the trajectory of CAVs in a mixed flow scene to pass traffic light intersections.The framework designs multi-objective reward functions for vehicle fuel consumption,traffic efficiency and safety.For the trajectory broken line problem in the existing model,a delayed reward strategy is proposed to restrain the problem.The output acceleration of traditional reinforcement learning model is unstable.In this paper,the jerk is taken as the output action instead of acceleration,and the reward function is used to control the vehicle to improve the comfort of passengers.The experiments result show that the proposed model can smooth the vehicle trajectory effectively compared with the traditional model and other reinforcement learning models.When the penetration rate of Cavs is 40%,the performance of vehicle fuel consumption,traffic efficiency and safety are improved by14.8%,10.9% and 1.8% respectively.In addition,this paper also analyzes the influence of Cavs on traffic flow under different Cavs permeability.Experiments show that Cavs can basically eliminate traffic oscillation when Cavs permeability is about 60%.CAVs personalization strategy is taken into consideration,a multi-agent reinforcement learning framework is proposed to replace the original shared strategy model.Each CAVs model has its own independent agent decision-making network,which designs reward functions that conforms to the corresponding strategy,and trains multiple agent decision-making networks following the ideas of centralized training and decentralized execution.Experiments show that the introduction of multi-agents,while considering the overall traffic flow performance balance,different CAVs have different performance advantages.In order to simulate a more realistic traffic scene,this paper analyzes the disturbance characteristics of HVs,and the model’s adaptability to HVs instability was studied.A Monte Carlo trajectory simulation strategy is proposed to estimate the probability distribution of the estimated time to reach the traffic light,solved the problem of inaccurate estimated time caused by the disturbance of HVs,and improve the accuracy of the reward value,thereby the adaptability and robustness of the model are improved.The experimental results show that when the disturbance = 1,the number of red light running and the number of broken track are reduced by 47.2% and 40% respectively. |
