Research On Regional Traffic Optimization Method Based On Q-learning Evolutionary Game Model

The purpose of regional traffic optimization is to balance the flow between intersections and alleviate traffic congestion in the region through coordination between intersections.Whether it is traditional regional coordinated control or regional traffic optimization that combines reinforcement learning and game theory,existing research basically uses specific traffic parameters to build game models or reinforcement learning models to achieve regional traffic coordinated control at the microscopic traffic simulation level.The existing problem is that the intersection conflict under the macroscopic road network structure is not combined with the microscopic traffic signal optimization.Therefore,this paper constructs the transportation network Q-learning model and the game network Q-learning model,and combines the two-layer model to propose a regional traffic coordination optimization method based on the Q-learning evolutionary game model.This method uses individual game strategies in the game network to guide the selection of signal strategies for intersections in the actual road network,thereby realizing the combination of macroscopic road network evolutionary game and microscopic intersection signal optimization.This paper has mainly achieved the following work:First,this paper uses evolutionary game theory to analyze intersection conflicts and establishes an evolutionary game model of the prisoner’s dilemma at intersections.Consider optimizing the area or intersection from a global perspective,define the global attributes of the intersection based on the betweenness,and introduce the global attributes of the intersection into the heterogeneous learning ability of the intersection.Relying on the regional traffic network structure,this paper analyzes the impact of the learning ability of intersections as individuals in the evolutionary game on the cooperative behavior of intersections in the region from a macro perspective.Secondly,aiming at the problem that the income of adjacent intersections and the game strategy cannot be obtained at the intersection in the game process,this paper proposes the intersection evolution game model based on Q-learning.Three decision-making mechanisms of Q-learning are used as the selection mechanism of the intersection game strategy in the evolutionary game.Experiments are carried out for different complex network types,game model parameters and reinforcement learning parameters,and the changes in the level of network cooperation under different conditions are quantitatively analyzed.Finally,this paper establishes the Q-learning model of the actual traffic network,and combines it with the research of the intersection Q-learning evolutionary game in the game network,and proposes a regional traffic signal optimization algorithm based on the Q-learning evolutionary game model.Based on SUMO-Python,a Q-learning evolutionary game traffic interactive simulation platform is built,and different algorithms are used to train the signal control strategy of the road network to obtain a real-time signal optimization model.Simulation experiments show that,compared with traditional reinforcement learning control and timing control,the algorithm improves the average speed of the road network and relieves traffic congestion in the area.