| With the rapid development of the national economy,the scale of the city continues to expand,the number of population and vehicle flow continues to increase,and the traffic problem has become increasingly prominent.In urban intersections,people use traffic lights to control the traffic of vehicles,and adopt the way of fixed time length of traffic lights.Under different traffic flow conditions,the average waiting time of vehicles will be too long,which will lead to vehicle congestion and affect the overall road traffic situation.Therefore,how to choose a reasonable way to control the traffic lights is very important to reduce the congestion of intersections.In this thesis,we mainly use reinforcement learning to control traffic lights.Firstly,in the ideal scene,two road modes are set up through sumo: one is two-way four lanes at a single intersection,and the other is two-way four lanes at multiple intersections.Two vehicle flow modes are configured: one is the mode of constant vehicle flow,and the other is the mode of regular vehicle flow changing with time.In addition,the map of real scene and vehicle flow are added.Secondly,in reinforcement learning,state space,action space,reward function and evaluation index are redefined.The state space is a function of the proportion of the queue length of the stationary and non-stationary vehicles in the road,the action space is a function of the four phases of the traffic lights,and the reward function is a function of the average waiting time of the vehicles.In a single intersection,the evaluation index is the average waiting time of vehicles in all lanes.In multiple intersections,the evaluation index is the average waiting time of vehicles in all intersections.Then,five scenarios are set,namely,the vehicle flow at a single intersection is constant,the vehicle flow at a single intersection changes,the vehicle flow at multiple intersections is constant,the vehicle flow at multiple intersections changes,and the real scene of multiple intersections.Finally,we choose five ways to simulate: fixed traffic light time length,semi fixed traffic light time length,Q-learning algorithm,strategy gradient algorithm and A3 C algorithm.The experimental results show that the average waiting time of A3 C algorithm is less than that of the other four modes.It is verified that A3 C algorithm can reduce the average waiting time of vehicles in different intersections and different vehicle flows,alleviate traffic congestion,and has high efficiency and superiority in traffic signal scheduling. |
