Research On Train Scheduling Method Based On Multi-agent Reinforcement Learning

Train scheduling problem is an important research topic in the field of railway transportation.Efficient and reasonable train timetable can not only alleviate the traffic pressure to the greatest extent,but also effectively reduce the commuting time and resource consumption of trains.Train scheduling problem in complex railway network is a classic NP hard problem.For decades,the attempt to find the optimal scheduling scheme by using traditional algorithms has never stopped.Dynamic programming,genetic algorithm and other mathematical methods have achieved remarkable results in this field.On the other hand,reinforcement learning(RL)theory has developed rapidly in recent years,and has been widely used in various decision problems.In particular,multi-agent reinforcement learning(MARL)plays an increasingly important role in the complex environment of multi-agent interaction.In this paper,the method of multi-agent reinforcement learning is combined with the train scheduling problem,and a new multiagent reinforcement learning model framework is proposed according to the train operating environment.The strategy of train scheduling is optimized continuously by using the exploration and self-learning of agents in the railway network,so as to achieve the purpose of reasonable train scheduling.In this paper,the following two multi-agent reinforcement learning(MARL)frameworks based on different decision strategies are proposed to solve the train scheduling problem.Synchronous Cooperation Network is a train scheduling method based on Synchronous decision.In the environment,agents(i.e.,trains)interact with each other in the form of cooperation.In this method,agents adopt the strategy of centralized training and decentralized decision-making.In the process of training,agents can exchange part of the information,so that they can get a wider range of observation information.When making decisions,agents are relatively independent and act according to their own strategies.In the cooperative environment,the agents are equal,which is more conducive to the convergence of the overall strategy to the global optimal direction.A train scheduling method based on asynchronous decision,namely asynchronous cooperative network.According to the different attributes of trains in the environment,different priority levels are set for them.The agent with high priority makes the decision first,and the agent with low priority makes the decision later.Thus,an agent with a low priority can see the actions that the agent with a high priority might make,and take this information into account as an influence on the decision.In this model,the observation information of each agent is different,and it adopts decentralized training and decentralized decision-making.The experiment shows that the advantage of asynchronous decision-making is to reduce the probability of the collision between the agent and the vehicle transfer at the intersection.Generally,the train with higher priority will pass first,which saves the time of vehicle transfer.Finally,in order to show the application results of multi-agent reinforcement learning in train scheduling problem more intuitively,the relevant prototype system is introduced in the end of thesis.The prototype system combines environment setting,model setting and results display together.