Urban Regional Traffic Signal Control Approach Driven By Cooperative Max-Pressure

Urban regional traffic signal control is used to analyze the traffic environment conditions of intersections which can generate traffic signal control strategies automatically.Accurate traffic signal control strategy provides a logical control point for urban traffic management and a good road network experience for drivers,which is a crucial research direction in the field of intelligent transportation.Recently,reinforcement learning has become an effective way to improve regional traffic signal control.But in real traffic scenario,as the scale of regional road networks increases,the traffic flow becomes more and more complex,leading to an exponential growth in the dimensionality of the state-action space of reinforcement learning agent.The effectiveness of deep reinforcement learning in decision making and feature selection provides a novel way to solve regional traffic signal control problem.However,existing approaches rely on the subjective experience of the researcher in selecting parameters for the reward design,and tuning of the parameter weights can lead to sensitive travel time.In addition,there is still a lack of perception in the regional traffic environment intersection state pressure information,which restricts the performance improvement of the agent to control traffic signals.In summary,the thesis studies the regional traffic signal control approaches based on deep reinforcement learning driven by transportation control theory cooperative max-pressure.And the main research work are as follows:(1)The current popular regional traffic signal control approaches are investigated and experimented,and the effects of traditional and deep reinforcement learning models on traffic signal control strategies in terms of average travel time and road network throughput are evaluated.The deep reinforcement learning model outperforms the traditional model on both synthetic and real traffic flow datasets,which demonstrates the better adaptation of the deep reinforcement learning model to dynamically changing traffic flows.Therefore,the thesis selects the deep reinforcement learning model as the base model to provide the basis for the subsequent study.(2)Aiming at the design of reward function in traffic signal control agent which rely heavily on subjective experience,a traffic signal control model based on cooperative max-pressure reward named CMPLight is proposed.Firstly,a reward function with transportation control theory support is designed and a theoretical analytical proof is given,while extending the intersection environment state to downstream intersections.Compared with the results of other models,CMPLight can significantly reduce the average travel time on arterial regional traffic networks.For example,the average travel time is reduced by 31.26%and 29.96%on the synthetic and real traffic flow datasets of the arterial regional road network,respectively,and by about 26%on the synthetic traffic flow of the simple grid regional road network.It also shows better adaptability and stability for different sizes of arterial regional traffic road networks.(3)Aiming at the inadequate perception of intersection pressure information in the regional road network traffic environment,this thesis proposes an agent named CMPLight~+for traffic signal control based on cooperative max-pressure state.Firstly,the intersection pressure information is introduced in the environment state definition,and the influence of neighboring intersections on the target intersection is learned by the graph attention mechanism to obtain the current intersection spatial state feature representation.Then,a temporal convolutional network is used to capture historical state information,which in turn yields a historical spatial and temporal state representation of intersection.With arterial and grid area traffic road network data,CMPLight~+performs to reduce average vehicle travel time and increase road network throughput compared to existing approaches.For example,the average reduction in travel time on the arterial regional road network synthetic traffic flow dataset is 5.80%compared to CMPLight,while having optimal road network throughput.