Research On Intelligent Traffic Signal Control Method Based On Deep Reinforcement Learning

Traffic congestion has seriously affected the healthy and sustainable development of global economy and environment.As the key node of urban traffic network,intersection is the main area where traffic congestion occurs,so how to improve the traffic signal control efficiency of intersection is the key to effectively alleviate urban traffic congestion.Traditional traffic signal control mainly optimizes traffic signal control parameters from two aspects: rules and models.There are many ideal assumptions that do not conform to the actual traffic conditions and the control effect is poor.In recent years,with the development of big data technology to obtain rich and diverse traffic data,the enhancement of computing power and the maturity of artificial intelligence technology,data-driven traffic signal control method has become a new research direction,among which the optimization method of traffic signal control combined with deep reinforcement learning is the most important research hotspot.Although many traffic signal control optimization algorithms based on deep reinforcement learning have been proposed and achieved good results.However,the existing control algorithms still have the following shortcomings:(1)The research on signal control of single intersection mostly adopts structured traffic state representation,which has some problems,such as insufficient accuracy of information description and insufficient perception ability of traffic state nodes.(2)The existing signal control research based on deep reinforcement learning mostly uses the current traffic state for training,ignoring the time sequence characteristics of traffic flow,and the control effect is limited when the traffic flow changes greatly.(3)In the research of traffic signal control at multi-intersection of regional road network,there are some problems,such as too high dimension of traffic state under complex road network and too long time for coordinated communication among agents,which seriously affect the traffic efficiency of road network.In view of the above problems,the research work carried out in this paper is as follows:1.A traffic signal control method based on heterogeneous graph deep reinforcement learning with double attention mechanism is proposed.We combine heterogeneous graph neural network with reinforcement learning based on node-level and semantic-level attention mechanism,and through graph neural network’s powerful processing ability in non-European spatial data,we can mine the potential correlation characteristics in traffic nodes and automatically pay attention to important state components to enhance the network’s perception ability,and provide potential information support for reinforcement learning decision reasoning and state prediction to make accurate signal control decisions.Experimental results show that,compared with other advanced algorithms,the algorithm proposed in this paper has improved in many traffic performance indexes.2.A traffic signal control method based on BGRU traffic flow prediction deep reinforcement learning is proposed.Firstly,a concise and efficient traffic state is designed by using the unique heat code to accurately depict the current traffic state.Secondly,according to the current traffic state,the two-way gated circulation unit is used to predict the future traffic situation,and the predicted traffic state at the next moment is used as the augmented information and the current traffic state as the input of the signal control algorithm.Finally,the DQN-based deep reinforcement learning algorithm is used to make the optimal control decision of traffic signal for the augmented traffic state combining the current and future traffic information.Experimental results show that the proposed algorithm is superior to the traffic signal control benchmark algorithm based on deep reinforcement learning in many traffic performance indexes.3.A deep reinforcement multi-intersection signal control method based on digital twinning is proposed.Firstly,we set up a four-layer digital twin architecture of traffic network signal control at multiple intersections.Through real-time information transmission and data fusion,we can realize the mapping of real traffic network from physical space to digital space,and build the digital twin of traffic network.Secondly,the digital twin model based on graph neural network deep reinforcement learning is modeled to realize the virtual-real interaction between real traffic network and digital twin traffic network.Finally,with the maximum traffic efficiency of the traffic network as the optimization goal,a single agent is trained to control the global signal of the multi-intersection road network.Experimental results show that the proposed algorithm is superior to the benchmark algorithm of multi-intersection traffic signal control based on deep reinforcement learning in many traffic performance indexes.