Research On ATO Control Algorithm Of Rail Transit Based On Reinforcement Learning

The Automatic Train Operation System(ATO)is the most important part to ensure the control level of train operation.Its primary goal is to adjust the traction and braking force of the train in real time according to the different operating environments of the train,so that the train can operate safely,reliably and efficiently according to the agreed instructions.Train operation is a very complicated process.The paper focuses on the complex characteristics of train system nonlinearity,multivariate,etc.,comprehensively considers the train operating environment and train control characteristics.Based on reinforcement learning,non-parametric learning,dynamic optimization,end-to-end learning,and self-training,a train automatic driving control system algorithm for improving the generalization ability of fuzzy control ideas is proposed.Through deep reinforcement learning,the system automatically generates a set of fuzzy control schemes for the existing train model,the required target speed curve and the demand control strategy,which replaces the design of traditional expert experience and improves the universality of fuzzy control.The main research contents of the paper include the following points:1.Research on the function,basic structure and traditional control algorithm of train automatic driving system,abstract the problem of train control problem,and establish a simplified model of train operation process.2.The related application of reinforcement learning in control algorithm,the principle and structure of DQN algorithm are studied,and the design of train automatic operation control algorithm based on reinforcement learning is completed,including train simulation environment simulation,target curve design and train input status,graphic design,convolutional neural network structure design,State output control,reward strategy,action strategy,DQN training parameter design and so on.Aiming at the shortcomings of traditional train ATO control algorithm and the statistics and analysis of the running process data of Hangzhou Metro Line 4,the DQN algorithm based on discrete control is selected,and the image representation method in deep learning is introduced to describe the state space,using classical convolution.The neural network obtains the corresponding output state,and then trains the convolutional neural network through reinforcement learning to make the optimal output according to the designed strategy.The method generates an optimal control strategy for the given target by interacting with the simulation model environment,and gets rid of the traditional train control algorithm based on the traditional parameter identification model,and realizes the precise tracking of the train speed.3.In view of the shortcomings of DQN algorithm in train control problem,this paper adds step control strategy to improve the control effect of original DQN algorithm based on DQN algorithm.This method greatly enriches the state space of control output.The control effect is obviously improved.Also,the convergence speed is accelerated,and the precise tracking of the train speed is realized.