Research On Deep Reinforcement Learning For Ecological Urban Traffic Signal Control

Urban traffic congestion wastes a lot of travel time,seriously aggravates exhaust emissions,and causes economic losses.Traffic Signal Control(TSC)at intersections can effectively alleviate traffic congestion by reasonably planning the traffic flow of road networks,so it plays an important role in solving traffic congestion problems.Since traffic signal control is essentially a sequential decision-making problem,most of the current research work on traffic signal control using Multi-agent Reinforcement Learning(MARL)model has been extended from traffic signal control at a single intersection to global control at multiple intersections,and certain results have been achieved.However,there are some shortcomings in the existing research.First,the existing urban traffic simulation environment is not realistic.It cannot be close to the real-world traffic conditions,resulting in the inability to ensure the validity of actual road tests.Secondly,most of the existing studies on traffic signal control in MARL focus on designing effective communication methods,but ignore the importance of how intelligent Agents interact in cooperative communication,and the communication between Agents in MARL traffic signal control has not been studied deeply enough.Finally,most of the existing traffic signal control algorithms are in the consideration of economic interests,and rarely consider the ecological traffic concept.Regarding the issue above,this paper takes the traffic signal control of a single intersection as the starting point,and then deepens the traffic signal control of multiple intersections based on deep reinforcement learning,and carries out a detailed study with the following results.(1)This paper compares the existing traffic simulation software and conducts the secondary development of SUMO simulation platform,including the construction of synthesis and real traffic network construction,and then sets up the real traffic flow in SUMO based on the traffic flow data of a certain time period made public by the traffic management department,and builds a large-scale urban road network traffic simulation environment close to the actual physical scenario to provide experimental guarantee for the actual traffic signal control problem in theoretical research.(2)To address the TSC problem at single intersections,this paper proposes the Fuel-ECO TSC model to improve the traffic efficiency at isolated intersections.The approach utilizes Deep Reinforcement Learning(DRL)techniques to sense the high-dimensional traffic state in real time and efficiently adjust the traffic signal control strategy.The TSC strategy for controlling multi-objective traffic signals is described in the Agent design.Based on the improved adaptive traffic signal control strategy,this method provides the best speed curve for approaching vehicles to smooth the traffic flow and improve the fuel economy of vehicles.(3)For the TSC problem at multiple intersections,this paper constructs the Graph Cooperation Q-learning Traffic Signal Control(GCQN-TSC)model,where graph collaborative attention enables the Agent to adjust its attention in real time based on dynamic traffic flow information in The graph collaborative attention allows the Agent to adjust its attention in real time according to the dynamic traffic flow information and perceive the traffic environment quickly and effectively in a larger range.Moreover,DGQ(Deep Graph Q-Learning)algorithm is proposed in this model to extract the spatio-temporal features of different traffic scenes and provide the optimal signal phase for each intersection.Meanwhile,this paper creatively integrates the ecological traffic concept into MARL traffic signal control,which is dedicated to reducing traffic exhaust emissions.Finally,experimental results conducted using the SUMO traffic simulation experiment platform show that GCQN-TSC outperforms other traffic signal control methods in terms of performance indicators such as average queue length and waiting time.