Research On Network-Wide Traffic Signal Control Model Based On Reinforcement Learning

With the rapid progress of urbanization in China,the number of motor vehicles continues to increase,and the congestion problem caused by the sharp increase in urban traffic pressure is becoming more and more serious.Traffic signal control can effectively solve the conflicts of traffic flows within the intersection,and sort out the right of way for vehicles in each direction.The joint traffic signal control for all signal intersections in the traffic network can improve the efficiency of traffic operation from the overall level of the traffic network,promote the smooth and rapid passage of vehicles through the traffic network,and greatly solve the problem of urban traffic congestion.From the perspective of data-driven control strategy,this paper uses reinforcement learning to self-learn excellent control strategy characteristics in the process of continuous interaction with the environment,expands the control rules and forms of multi-agent in the traffic network,and develops the traffic network intersection signal control model based on reinforcement learning.According to the differences of intersection signal control modes,the research is carried out under step and periodic control modes respectively,and tries to solve the difficulties in data acquisition,information transmission,model calculation and other aspects existing in the application of actual control based on reinforcement learning model.The attention mechanism is integrated into the reinforcement learning model structure to improve the intersection ability to explore the key information of multi-scale traffic network data.The central training and decentralized execution model framework is adopted,and agents make use of historical states for policy making,providing sufficient time guarantee for the decision-making process.The setting of state-reward-action composition with different information complexity highlights our model abilities in the perception,prediction and decision-making under the state of information loss,and a method for predicting traffic state of downstream intersections using intersection exit data is proposed.The periodic control mode studies the continuous-discrete hybrid action space model,and specially constructs the reinforcement leaning formulation to realize the periodic control objective model expression that reduces the average waiting time of vehicles as much as possible without generating multiple stops.A traffic simulation platform for traffic control is developed to provide standardized training and testing scenarios for the developed algorithm model.Evaluation indicators are formulated from multiple perspectives of driving experience and traffic network performance.And experimental comparisons are made with a variety of advanced control algorithm models that have been previously proven to work.Experimental results show that the model constructed in this paper surpasses the comparative model in all aspects of indicators,and the speed spatialtemporal distribution results explain the model superiority in that it can quickly evacuate traffic congestion,control the scope of congestion area,and maintain the smooth and balanced operation of vehicles in the traffic network.