Research On Path Planning Based On Multi-Agent Cooperative Communication Learning

With the rise of online e-commerce and the physical industry,unmanned warehousing based on automatic guided vehicles(AGV)has gradually been widely used.The three generations of ASRS intelligent warehousing systems and KIVA systems rely on AGVs to pick-up and deliver.All AGVs in the unmanned storage scenario can form a typical multi-agent system,and the AGV scheduling process is modeled as a classic multi-agent path planning problem.To be specific,each agent has its corresponding unique destination,the algorithm needs to plan the optimal path for each agent to reach the destination,it must satisfy the constraints that the agents do not collide with each other during the movement process.The classic multi-agent path planning algorithms(such as ODr M*,etc.)is a centralized search algorithm based on A*,but when the scale of the problem increases or encounters a non-stationary environment,such static planning methods will face excessive computational overhead and require multiple times.The problem of repeated planning,its scalability is weak.Therefore,academia and industry are more and more inclined to seek a decentralized dynamic strategy,which can dynamically output the movement behavior of the next moment according to the local observation information at the current moment.In recent years,a lot of studies related to path planning based on decentralized multi-agent reinforcement learning has been proposed,but the current mainstream algorithms have the following three common problems: 1)The decentralized planning results are still improved compared to the optimal planning results2)The conflict caused by decentralized decision-making is difficult to avoid,and artificial post-processing is widly used to avoid it? 3)The relevant training of the path planning algorithm in large-scale scenarios is very time-consuming and difficult.In response to the above problems,the contributions of this paper is summarized as follows:1.For the combination of decentralized path planning strategy and communication mechanism,this paper designs a general communication multi-agent enhanced path planning algorithm,and compares the effects of different communication designs on the path-finding strategy through experiments,which provides a theoretical and experimental basis for subsequent work.2.For the conflict between agents,this paper proposes a multi-agent reinforcement learning method(Pr Ioritized COmmunication learning method,PICO)combined with priority communication.By learning the priority information from the expert strategy,learning the overall optimal priority allocation,and constructing a dynamic decentralized topology communication architecture based on the priority information,so as to realize the communication learning with cooperative avoidance ability.3.In view of the challenges in large-scale scenarios,this paper introduces the idea of mean field based on the PICO algorithm,and integrates the observation information in large-scale test scenarios into observations in small-scale training scenarios,so that the results obtained from small-scale scenario training The strategy can be adapted to a large-scale test environment.At the same time,the large-scale reinforcement learning framework MAgent is used to optimize the inference efficiency.Finally,it is verified by experiments that the performance of the algorithm can still have stable performance in large-scale test scenarios.In this paper,the related algorithms are trained and tested in a two-dimensional gridded scene,and various settings of obstacle density and number of agents are tried.The experimental results show that the performance indicators of the proposed algorithm are better than other baseline methods,while ensuring lower collision rate and better scalability.