Traffic Signal Control Method Based On Multi-Agent Reinforcement Learning

With the increase of population and the improvement of people’s quality of life,the number of private cars has increased sharply,which has led to serious traffic congestion,which has brought huge losses to people’s lives and the economy.The key to alleviating traffic congestion is to optimize the control strategy of traffic lights.Traffic light control has been a very challenging problem in this field,especially in large urban road networks.In recent years,multi-agent systems(Multi-Agent System)have been widely used to model intelligent transportation systems,and multi-agent reinforcement learning algorithms have become a popular method for solving multi-agent system problems.The use of Holonic-based organized multi-agent systems can further reduce the complexity of large-scale systems.When using the multi-agent deep reinforcement learning method to solve the signal light problem,we will encounter the following problems:(1)The increase in the number of agents will lead to an exponential increase in the action space and an explosion in dimensions.(2)Multi-agents jointly affect the environment,making the environment unstable,and the agents cannot Learn to cooperate to dredge traffic flow.(3)The large scale of urban road network,multiple intersections,and a large number of signal lights lead to the large scale of multi-agent systems,which is difficult to model.To solve the above problems,this paper works as follows:This paper first uses SUMO(Simulation of Urban Mobility)to model the traffic environment.Among them,it is necessary to model the road network,traffic flow and traffic lights at intersections to simulate traffic lights and direct the traffic flow in the road network.In this paper,the MADDPG(Multi-agent Deep Deterministic Policy Gradient)algorithm is used to optimize the control strategy for small-scale road networks.This algorithm can well solve the problem of dimension explosion and environmental instability,and can enable agents to learn to cooperate with each other.Ability.Experiments have proved that in the regional road network,the MADDPG algorithm has obvious advantages over the fixed timing strategy and the IDQN(Independent Deep Q-Learning)strategy.After verifying the effectiveness of the MADDPG algorithm in small-scale road networks,this paper further studies the possibility of modeling large-scale traffic networks through the organization of Holonic Multi-Agent System.The traffic network containing81 intersections is divided into multiple subregions,and abstract hyperholograms are assigned to control each region.The hologram is divided into two levels.The signal light agent at the intersection is located at the first level,and the MADDPG algorithm is still used for local strategy optimization.The hyperhologram is at the second level,which is an abstract concept without a physical entity.The holograms at this level are connected to each other The improved maximizing pressure method(Max Pressure)was used during the period.The inter-level interaction between the holograms in the two levels is helpful for dredging large-scale road networks.The experimental results show that the combination of the MADDPG algorithm and the multi-agent system based on the holographic system can effectively prevent the oversaturation of road network vehicles,reduce the average delay time and average queuing vehicles,and improve the vehicle capacity of the road network.average delay time and average queuing vehicles,and improve the vehicle capacity of the road network.