Research On Smart Traffic Signal Control Method Based On Semi-supervised Double Dueling Broad Reinforcement Learning

With the development of the economy and the increase in population density,the number of vehicles in the country is increasing,and the problem of urban traffic congestion cannot be ignored.One of the issues that local governments hope to solve first is to develop effective smart traffic signal control system to alleviate urban congestion.Traditional traffic signal control system mostly adopts relatively simple procedures,and cannot adjust the green light time according to the real-time traffic flow.In the face of fast-growing traffic flow,it has obvious limitations.Therefore,it is very meaningful to establish an effective traffic signal control system that can adapt to the real-time changes of urban traffic flow and achieve optimal control of traffic flow.This thesis proposes a semi-supervised double dueling broad reinforcement learning to support traffic service in smart cities,and designs different experiments to verify the effectiveness of the algorithm.The main content of this thesis includes the following aspects:(1)A traffic simulation platform based on SUMO is built,which can customize road conditions and driving routes of vehicles.Completed the formulation of traffic rules in this simulation environment,and defined the three elements of reinforcement learning in the traffic scene.The vehicle position information on the road was used as the state input.The action space adopts an unfixed phase sequence to improve the flexibility of the model.The reward function is defined as the difference between the accumulated waiting time of the vehicle between two adjacent moments.In addition,the traffic flow under different conditions is set to lay the foundation for the experimental design later;(2)The double dueling BRL(DDBRL)algorithm frameworks are proposed.By optimizing the network structure and action value function,it solves the problems of overestimation and slow convergence in the traditional deep/broad reinforcement learning when encountering practical problems.The experimental results show that in low-density traffic scenarios,compared to the DRL and BRL approaches,our DDBRL approach can effectively reduce the average waiting time of vehicles by 26.8% and 8.7%,showing better performance.In addition,in the two different scenarios of high-density traffic,when the number of vehicles further increases,the road becomes congested,the scene is complex and contains more uncertain factors,the traditional fixed-light method has completely failed,and it needs to be improved accordingly.Even in complex scenarios,the control method based on reinforcement learning can still intelligently optimize and control traffic signals according to the current state of the intersection.Finally,by observing the actual control signal,the rationality of the proposed control method is further verified;(3)A semi-supervised double dueling broad reinforcement learning(semi-DDBRL)is proposed to support traffic service.The proposed method is a further improvement based on the DDBRL,combined with pseudo-label-based semi-supervised learning,the problem of small data sets in actual traffic control scenarios is solved.In addition,the influence of the random factor of the BLS weight,the different number of BLS network nodes and the different unit time of the green phase on the performance of the proposed algorithm is also discussed.Based on the experiments,when there is a large amount of unlabeled data in traffic scenes,the proposed semi-DDBRL approach achieves better results.Compared with the original BRL method,the proposed approach effectively reduces the average waiting time of vehicles by 11.7%.