Multimodal Traffic Adaptive Signal Control Based On Deep Reinforcement Learning

Efficient real-time traffic control system significantly benefits economy and environment.In recent years,there is a growing interest in using reinforcement learning technique for adaptive traffic signal control.Numerous reinforcement learning based methods design their reward function to reduce vehicle’s delay and queue lengths,or to maximize the throughout and speed of vehicles.Their results have shown that it can significantly improve vehicle’s mobility.Despite their promising results on traffic efficiency,existing studies do not yet consider the benefits of multimodal traffic in the real-world situation.Most of the existing studies focus on the efficiency of cars and treat different modes of traffic with the same priority.If only delay is used as the reward to train reinforcement learning based traffic signal controller,it contains limitations.There will be a higher waiting time to the direction which have light traffic.This will reduce the driver’s comfort and pedestrians will choose to run a red light,which may cause more serious problems.To solve these problems,this paper proposes a new adaptive traffic signal control method based on reinforcement learning to coordinate the multi-mode traffic interests at intersections.The goal of this method is to reduce the per capita waiting time of different traffic modes,including vehicle,bus,pedestrian and non-motor vehicle.This method gives people the same priority when pass the intersections.The rewards,states,and actions of the model are then designed to balance the efficiency of the multimodal traffic.The main research contents of this article are summarized as follows:(1)We propose an adaptive signal control method for multimodal traffic at a single intersection based on deep reinforcement learning.The position and queue length of vehicle,bus,pedestrian and non-motor vehicle are used as the state input.Actions are phases that should be taken next.The objective of the agent is to maximize the total wait time reward.In order to avoid a small group of cars waiting for too long,the punishment mechanism is established into the model.Simulation experiments have shown that the trained agent can greatly improve the mobility of multimodal traffic at intersection.The average waiting time of each mode decreases by more than 10% compared with the traditional traffic signal control method.(2)We propose a coordinate signal control method for multimodal traffic in arterial based on multi-agent reinforcement learning.This Agent is designed based on the single intersection agent mentioned above.It builds a cooperative multi-agent reinforcement learning framework by using centralized training and distributed execution.The framework considers the communication between intersections with signal phase information.The agent aims to maximize the per capita waiting time reward of the whole arterial.The simulation results have shown that the proposed model performs better compared to the green wave model.(3)A multimodal traffic scenario for training and testing was established based on SUMO software.In this simulation environment,weibull distribution and normal distribution are used to generate traffic flow of vehicles,pedestrians and non-motorized vehicles.Using headway data to randomly generate the bus flow.It is a more real and comprehensive restoration of the real intersection situation.In the simulation environment,many python functions are provided to control the signals or to extract simulation data.