Research On AGV Material Distribution Strategy Under Uncertain Environment Based On Deep Reinforcement Learning

As a key link in the workshop production process,timely,accurate and agile material distribution is an important prerequisite to ensure the continuous and stable operation of workshop production.However,there are various uncertainties in the production process of discrete manufacturing workshops.Uncertainties in the material demand stage(equipment failure,workpiece rework,etc.)lead to uncertainty in material demand time,and uncertainties in the material delivery stage(temporary blockage of path,AGV failure,etc.)lead to uncertainty in material delivery time,which significantly increases the complexity of the whole material distribution chain.Therefore,to address the problems of inefficiency,weak dynamic responsiveness and insufficient real-time decision making in material distribution in discrete manufacturing shop floor under uncertain environment,this paper focuses on the research from material distribution real-time optimization strategy,and then realize the just-in-time material distribution.The main research contents and contributions of this paper are as follows.(1)The perturbation factors in the workshop under uncertain environment are analyzed and summarized.Considering the dynamic disturbances in material demand and delivery phases,a dynamic time window of material demand and path resistance coefficient calculation model are established.The dynamic time window is used to characterize the disturbance in the material demand stage and the path resistance factor is used to characterize the disturbance in the material delivery stage,so as to quantify the dynamic and stochastic shop floor production environment more accurately.(2)For the needs of accuracy,timeliness and self-adaptability of material distribution in the shop floor under uncertain environment,the material distribution real-time optimization problem is defined as a semi-Markov decision process(SMDP),and its state space,action space and reward function are designed in detail.With the objective of minimizing material distribution cost and workstation penalty cost,a real-time optimization method of material distribution based on Deep Q Network(DQN)is proposed.And the feasibility and effectiveness of the method is verified under a single material distribution rule and single Automatic Guided Vehicle(AGV)condition with an actual machine shop as an example.(3)For the problem that a single material distribution rule cannot better adapt to the changing workshop environment,a real-time optimization method of multi-AGV material distribution based on Double Deep Q Network(DDQN)is proposed on the basis of the above research.The state space,action space(material distribution rules and AGVs)and reward function of the upper layer DQN are designed in detail.Finally,the feasibility and effectiveness of the proposed method are verified by example studies.(4)Based on the above theories and methods,a workshop material distribution simulation platform based on deep reinforcement learning is built,and the construction principle and construction process of the workshop material distribution simulation platform are elaborated in detail.Finally,the main functional interface of the simulation platform is shown with an example.