Research On Dynamic Scheduling Of Two YC In Automated Container Terminal Based On Deep Reinforcement Learning

With the continuous advancement of world economic integration,international trade has become more and more frequent.As an important hub for land-sea combined transportation,ports have gradually become the backbone of regional economic developme nt.In recent years,with the increasing throughput of ports and the continuous enlargement of ships,higher requirements have been placed on container terminals.Its technical level and management capabilities have become a new competitiveness beyond the geographical location and economic hinterland.At the same time,with the introduction of new concepts such as "green development" and "smart development" of ports,efficient and energy-saving automated container terminals have become an inevitable choice to deal with the new environment.Yard operation is an important part of the overall operation of the automated container terminal,which is closely related to the various operations of the terminal.It has a prominent influence on the complete operation process.Therefore,how to solve the operation scheduling problem of the yard equipment more efficiently and reasonably is the core of improving the overall operation efficiency of the terminal.Only when we resolve this point can we ease the pressure on terminal operations.At the same time,we can shorten the waiting time of ships and vehicles to improve the service level of the terminal to take advantage of the advantages of automated container terminals.Based on this background,this paper studies the real-time dynamic scheduling problem of two YC in a yard of an automated container terminal.This paper aims at the cooperative operation of two bridges in a single box area.It considers the mixed operation mode in which access box operation tasks exist simultaneously.At the same time,in consideration of uncertain factors such as the inability to cross between the bridges and the actual vehicle delays and random arrival of tasks,we aim to construct the Markov decision process model(MDP).We responded to the dynamic factors of the environment based on the idea of rolling time windows,and designed a quantitative method for the state of the environment to describe the dynamic environment in detail.In order to solve the problem that traditional Q-learning algorithms are difficult to deal with complex environments,this paper designs a DQN network for two YC dynamic scheduling based on deep reinforcement learning.At the same time,we improved the algorithm’s strategy exploration process and optimized sampling process.Finally,we analyzed the value range of key parameters in the algorithm structure through numerical experiments,and analyzed the effect of different optimized sampling strategies on the convergence of the algorithm.And we verify the computational performance of this method for solving the dynamic scheduling problem of two YC.The results show that this method can effectively carry out YC scheduling.Compared with the combination strategy,it can optimize about 10%-30%.It can also achieve a certain degree of optimization in terms of YC utilization and the proportion of no-load movement time.In terms of avoiding mutual interference of cranes and overtime waiting situations,it also shows certain optimization effects.And due to the characteristics of algorithm offline learning and online scheduling,the method can be applied to the real-time scheduling process,which has remarkable practicality.