Reshuffle Optimization In Container Terminal Based On Q Learning Algorithm

Port is the most important water and land transportation hub in the integrated transportation network,and the largest collection center in the logistics supple chain.With the deepening of goods containerization,the container throughput has increased in the terminal,the storage pressure of container yard is increasing,and the yard resources have become tense.In order to improve the quality of customer service and enhance the comprehensive competitiveness of the port,it is essential to optimize the scheduling management within the container terminal.The reshuffle ratio of the yard is one of the most important indicators of container terminal scheduling management.The reshuffle optimization during the picking process can effectively reduce the relocation probability.For the problem of reshuffle in container yard,firstly,the classification analysis is carried out from three cases: import container reshuffle,export container reshuffle and reshuffle when containers are moved.Then the paper analyzes the reasons for the reshuffle from three aspects: internal factors,external factors and other irresistible factors,propose some methods for reducing the amount of reshuffle from the aspect of yard management.Finally,it is determined that the research object of this paper is the import containers,and it is preferable to study the reshuffle optimization of the import containers to reduce the reshuffle ratio of the yard.The optimization goals of the reshuffle problem in the yard is to minimize the reshuffle number of a bay,and the prerequisite is that the storage status of the containers in a bay and the customers' extraction order of the containers are known.Based on this,the basic assumptions of the model are analyzed,the model variables are described,and the Markov decision process model for reshuffle optimization problem of the import containers in the container yard is constructed.In order to solve the reshuffle optimization model,a ?-greedy Q learning algorithm is designed.According to the influence degree of each factor in the yard on the reshuffle ratio,the key factors are selected to describe the multidimensional state space of the Q learning algorithm to reflect the system dynamics in real time.Determine the action set and reward and punishment system,after a certain relocation operation is completed,through the immediate return feedback of the pros and cons of the action.Through theoretical analysis,determining the change trend of learning factor,discount factor and exploration factor with the number of learning scenes,and designing the exploration strategy of action,in order to balance the convergence and overall result optimality of the algorithm.Finally,the examples of container bays with different scales is designed to verify the performance of the ?-greedy Q learning algorithm which is to solve the problem of reshuffle optimization of the import containers.The experimental results show that: 1)compared with the estimation formula of the number of rehandles from Kim,the optimization rate of the Q learning algorithm is more than 40%.2)Compared with the reference algorithm OH algorithm and IH algorithm,when the Q learning algorithm solves large-scale problems,the average optimization rate of the second reshuffle is 50% and 10% respectively;3)The solution result of Q learning algorithm of a single case with different scales is improved stability compared with the OH and IH algorithms.At most one of the 100 examples,the result is inferior to OH;at most four examples are inferior to IH.