Research On Intersection Traffic Signal Control Based On Deep Reinforcement Learning

Traffic congestion is a key factor that plagues the development of urban.Intersections are the key nodes of urban transportation networks and are also key areas prone to congestion.Unreasonable traffic signal control strategies at intersections are the main cause of traffic congestion.Traditional control methods cannot make realtime adjustments to dynamically changing traffic environments.With the progress of artificial intelligence technology,Deep Reinforcement Learning(DRL)methods are widely used in traffic signal control,providing new technical support for solving traffic congestion issues.DRL has the advantages of strong perception and a high degree of adaptability,but its decision-making ability needs to be further improved when applied to traffic signal control.Therefore,this thesis studies the DRL control strategy used in traffic signal control,and the main work is as follows:(1)To solve the problem of insufficient exploration and exploitation capabilities of agents due to greedy strategies in DRL,a priority Bayesian deep Q-network algorithm was studied.Adding a Bayesian linear regression layer at the end of the network structure updates the posterior distribution of the value function through Bayesian linear regression and executes Thompson sampling to select the optimal strategy in the posterior distribution,increasing the exploration of agents for actions with high uncertainty.A single intersection simulation environment is built based on the Simulation of Urban Mobility(SUMO).The simulation results show that the algorithm is an effective method to solve the problem of a greedy strategy unable to select the optimal signal phase.(2)The discrete traffic state encoding method for obtaining intersection state information requires detectors built into each discrete grid,which often leads to the use of some invalid state information and consumes computational resources.And in the actual traffic environment,implementation and deployment are difficult.To this end,a traffic signal control strategy for partially detection environments is studied.By simulating partially detection traffic environments and redesigning the reward function mechanism of the priority Bayesian deep Q-network.The simulation results show that the improved algorithm still has applicability under the same traffic flow conditions in partially detection environments.(3)In multiple intersections,Using all the state information at each intersection can lead to too much data to make decisions.Therefore,a weighted mean field traffic signal control algorithm for multiple intersections is studied.When using the state information of an intersection,a weighted mean field deep reinforcement learning method is used to simplify the information interaction between multiple intersections into a weighted mean effect between the central intersection and the neighboring intersection.The importance of the neighboring intersection relative to the central intersection is depicted through a self-attention mechanism,which reduces the dimension of state information while also taking into account the importance of state information.The simulation results show that the multi-intersection traffic signal control algorithm based on the weighted mean field deep reinforcement learning has more advantages in the collaborative control of traffic signals.