Research And Implementation On Traffic Signal Control Algorithms Based On Deep Reinforcement Learning

Applying advanced traffic signal control methods to scientifically control the traffic flow of the road network is one of the important technical methods to improve the traffic efficiency of the road network and alleviate traffic congestion.However,the existing traditional traffic signal control methods based on mechanism modeling still have certain limitations.In recent years,thanks to the rapid development of deep learning and reinforcement learning technology,deep reinforcement learning methods have gradually been introduced into the field of traffic signal control.Due to its infancy,there are still some theoretical research issues to be further improved and resolved.Therefore,this paper proposes some new traffic signal control methods based on deep reinforcement learning for single intersections and arterial roads.The main work includes the following three parts:1.Traffic signal control algorithm for single intersection based on Nature-DQN deep reinforcement learningAiming at the problem that existing traffic signal control methods based on deep reinforcement learning are difficult to timely update intersection signal control strategies using real-time traffic status information at intersections,this paper proposes a single intersection traffic signal control method based on improved deep reinforcement learning Nature-DQN.This method uses the number of real-time vehicles in each sampling time step to establish a reward function,which can effectively reflect the changes in current traffic status.The simulation test results based on SUMO show that compared with the traditional DQN based traffic signal control method,the proposed method reduces the average waiting time and queue length of vehicles by36.8% and 18.1%,respectively,and can effectively improve the traffic efficiency of intersections.2.Single intersection traffic signal control algorithm based on multi-step double DQN deep reinforcement learningTo solve the problem of slow training speed in existing traffic signal control methods based on deep reinforcement learning,an improved multi-step learning based traffic signal control method was proposed.This method establishes a new action space based on the original method,and introduces the Double DQN mechanism and multi-step mechanism to calculate the Q value to solve the problem of too slow training speed at the initial stage of training.The simulation results based on SUMO show that after adding a multi-step mechanism to the traditional DQN method,the average waiting time decreases by 21.8%,the average queuing time decreases by 17.1%,and the average driving speed increases by 7.6%.Therefore,the proposed method can effectively improve the control effect of the model.3.Coordinated control algorithm of trunk traffic signals based on improved multi-agent deep reinforcement learningAiming at the problem of excessive action space in existing traffic signal control methods based on deep reinforcement learning when solving the coordinated control of trunk lines at continuous intersections,a distributed trunk line coordinated control method based on deep reinforcement learning was proposed.The proposed method adopts a multi agent architecture,assigning independent agents to each intersection for traffic signal control.This method utilizes a state information sharing mechanism to share information between neighboring agents,and establishes a shared experience pool for multiple agents.Finally,simulation testing results based on SUMO show that compared to traditional distributed trunk coordination control methods,the proposed method reduces average waiting time by 33.7%,average queuing time by 11.1%,and average speed by 5.1%.Therefore,the proposed method can effectively improve traffic congestion and traffic efficiency of trunk lines.