Research On The 3D Container Loading Problem With Reinforcement Learning

With the rise of e-commerce and Internet economy in the 21 st century,the logistics industry is moving towards automation,networking,and intelligence from manual and mechanization.Intelligent Logistics focuses on the use of intelligent technologies such as artificial intelligence and the Internet of Things,so as to improve its decision-making and execution capabilities.This will greatly help to reduce the human intervention,to reduce the labor costs,and to improve the operational efficiency and management level of an enterprise.And in consideration of the high cost of logistics industry in China today,it is quite necessary to explore more scenarios where intelligent technologies may empower the logistics industry.For China,the world's second largest economy,a small increase in the utilization rate of resources can save a lot of costs.Therefore,this paper focuses on the problem of improving the utilization and seeks for a more intelligent and more efficient solution with lower labor costs,higher space utilization,and more profits.Thereby,greater economic benefits can be obtained.The three-dimensional container loading problem is to pack a set of rectangular boxes into a rectangular container so that the packed volume is maximized.Boxes cannot overlap with each other or exceed the container's boundary.Taking into account the cargo stability and the feasibility of operation,full-support constraints and rotation constraints are also considered.This paper proposed a two-stage three-dimensional container loading algorithm with Double Deep Q-Networks(Double DQN).At the first stage,Double DQN is used to train a model that can make decisions about how to take a packing action.At the second stage,the strategy of the trained model is used in a tree search method,called "lookahead tree",to further improve the space utilization of the container.Experiments show that the idea of applying reinforcement learning on three-dimensional container loading problem does make sense,and is tractable.This thesi has also verified the generalization of the algorithm on the real data set from a company.Compared with the classical greedy algorithm and the lookahead tree search algorithm,the space utilization has increased by 1.37% and 0.33% respectively by the Double-DQN-based Two-Stage 3D Container Loading Algorithm(TSCLA).It implies that it is promising for the TSCLA algorhtm proposed in this thesis to be widely applied in complex practices in the near future.