Research On Intelligent Traffic Signal Control Based On Reinforcement Learning

With the development of urbanization,traffic congestion has become a seri‐ous social problem.The development of efficient traffic signal control algorithms can not only reduce the economic losses caused by traffic congestion,but also improve people’s travel efficiency.As an efficient control optimization solution for complex systems,Reinforcement Learning（RL）has been adopted in traffic signal control increasingly.However,the performance of RL‐based traffic signal control methods relies heavily on accurate modeling of the traffic environment.Due to the limitation of traffic infrastructure,it is difficult to obtain real‐time ve‐hicle dynamic information in some traffic networks,which makes it difficult for existing RL‐based methods to adapt to the real traffic environment.Therefore,how to construct a high‐quality RL model for traffic signal control becomes a big challenge when the observation of traffic information is limited.In addition,with the development of V2 X technology,more and more vehicle dynamic informa‐tion（e.g.,vehicle speed and waiting time）can be obtained in real‐time.How to comprehensively use vehicle dynamic information to accurately model the state and reward in the RL model,and tap the potential of RL has also become a major challenge.To address the mentioned challenges,aiming to reduce the average travel time of vehicles and accelerate the convergence speed of the RL model,this paper proposes different traffic signal control methods for both two traffic scenes.The contributions are mainly as follows:（1）Aiming at traffic scenes where the observation of vehicle dynamic informa‐tion is limited,a new traffic signal control method based on intersection clustering is proposed.The method accurately models the traffic pressure based on the number of vehicles,and clusters the intersections according to the location and traffic flow,which can form multiple centralized control models for optimizing multi‐intersection traffic scenarios.Through traffic data sharing between intersections,this method can learn the efficient sig‐nal control strategy quickly.（2）For traffic scenarios with vehicle dynamic data obtained,this paper pro‐poses a novel intensity‐ and phase duration‐aware RL‐based method.Based on vehicle dynamic information,the method innovatively designs a “traffic intensity” concept and realizes accurate modeling of RL state and reward.The method also supports the selection of dynamic phase duration based on the number and speed information of vehicles,which greatly reduces the vehicle average travel time.（3）Based on the China Mobile One NET cloud platform,this paper develops a ”cloud‐terminal” collaborative traffic signal control system that supports the deployment of RL‐based methods.The system supports the real‐time acquisition of traffic data from the traffic environment and realizes real‐time traffic light control based on the RL‐based methods.Comprehensive experiments are conducted on multiple real‐world and sim‐ulated datasets of different road network scales to fully verify the effectiveness of the two methods.Compared with the latest traffic signal control methods,meth‐ods in this paper can significantly reduce the average travel time of vehicles and shorten learning time.