Research On On-line Reinforcement Learning Signal Control Method Based On DN Model

At present,the important problem of transportation in China is how to solve the problem of traffic congestion by using the current artificial intelligence technology,computer simulation technology and other high-tech under the condition that the urban space is limited and other constraints make the infrastructure construction difficult to expand.China’s intelligent traffic management and control started late,although there is a breakthrough in intelligent traffic signal control.However,in the actual complex traffic system,due to the nonlinear,uncertain and dynamic characteristics of the traffic system,many single point signal control systems with good effect can not play a very good role.Therefore,considering the coordinated control of multiple intersections and regional coordinated control has become a research hotspot at present,making the regional overall or multi intersection coordinated optimal is the key to solve the "strong coupling" problem of traffic signal control adjacent intersections.At present,the main research direction is how to use artificial intelligence and computer simulation to solve this problem.Therefore,this paper uses PTV-VISSIM traffic simulation software to build an online simulation platform,and uses reinforcement learning method to carry out research on intelligent traffic signal control.First of all,based on the ptv-vissim online reinforcement learning signal control platform construction,using the ptv-vissim COM interface and numerical simulation software MATLAB for secondary development to build an online simulation platform,according to the research needs to design different simulation environment.Develop the online reinforcement learning algorithm module,optimize the signal control scheme according to the real-time traffic conditions and evaluate the optimized signal control scheme.Secondly,an online reinforcement learning model of single intersection signal optimization based on prior knowledge is established.According to the traffic characteristics of single intersection,queue length and delay are selected as the traffic parameters of reinforcement learning,and "prior knowledge" is introduced into the traditional reinforcement learning model to make the model converge quickly.Then,the phase difference optimization of the adjacent intersections is carried out,and a heuristic reinforcement learning phase difference optimization model is established.It mainly constructs a double-layer learning network,the upper layer is BP(back propagation)neural network,the lower layer is Q learning model,and the "knowledge" learned by BP neural network is used to inspire the lower layer Q learning model.Finally,the DN(differentiated node)model theory is used to change the previous traffic signal control topology.Taking the signal control as the node and the intersection as the direction,a new reinforcement learning training method is proposed to change the minimum learning unit of reinforcement learning into two adjacent intersections.The model can ignore the topological structure of the road network,optimize the signal control scheme at different levels,and introduce the iterative training reinforcement learning method,so that the reinforcement learning training can be combined with the DN model,so that more The intersection reinforcement learning signal control can be expanded rapidly.