Research On Regional Traffic Signal Coordination Control Based On Multi-agent Deep Reinforcement Learning

Urban traffic congestion has become a serious social problem,which has a negative impact on public travel and social development.In order to relieve traffic pressure and improve traffic efficiency,regional traffic signal coordination control has become a hot topic in current research.Aiming at the insufficiency of the existing traffic signal coordination control research,this thesis propose a traffic signal control method based on multi-agent deep reinforcement learning.The algorithm adopts the deep double Q network model to solve traffic signal coordination control based on the collaborative multi-agent deep reinforcement learning algorithm.Simulation results show that this method can effectively reduce the average queue length and improve the traffic efficiency.The main contents of this thesis are as follows:(1)To solve the problems of traditional deep reinforcement learning algorithms such as overestimation and slow convergence,proposes a deep double Q learning algorithm for intersection traffic control based on priority empirical replay.The discrete traffic state coding method was used to transform the high-dimensional real-time intersection traffic information into a two-dimensional matrix composed of vehicle position and speed information.The phase pressure difference of intersection traffic was used as a reward function of reinforcement learning.The priority experience replay strategy was adopted to improve the utilization rate of training samples and accelerate the convergence speed of the algorithm.Taking single-intersection traffic signal control as the research object,the simulation results show that the proposed algorithm has faster convergence speed and better performance.(2)In the area traffic signal control,due to the complexity of road network,the coordination of control algorithm is highly required.A coordinated control method of area traffic signal based on cooperative multi-intelligence deep reinforcement learning is proposed.The regional traffic network modeling as an undirected graph,each node in the graph as a reinforcement learning agent,the single agent reinforcement learning traffic signal control algorithm is extended to the multi-agent control system,more intelligent by sharing between state and reward information to solve coordination control,through multi-agent markov game to achieve Nash equilibrium.Simulation results show that the proposed method can better coordinate and control regional traffic signals,improve the average speed of vehicles in the regional road network and reduce the number of vehicles queuing at intersections.(3)This thesis analyzes the application status of deep reinforcement learning in traffic signal control,and constructs a traffic simulation system for regional traffic coordination control experiment.The regional traffic network model is built in SUMO,the coordinated and optimized control algorithm is realized by Using Python,and the communication between the two is realized by calling TraCI interface.The system provides a simulation platform for traffic signal control algorithm.By modeling the actual traffic network and simulating the system with the deep reinforcement learning traffic signal control algorithm,the regional traffic coordination optimization control model and method proposed in this thesis can effectively reduce the regional traffic delay and improve the traffic efficiency of the road network.