Multi-agent Collaborative Path Planning And Formation Encircling Based On Reinforcement Learning

Multi-agent has a wide range of application scenarios at present and in the future,and gradually plays an important role in production,scientific research and daily life.The quality of the multi-agent collaborative path planning technology determines whether the agent can accurately reach the predetermined location to complete a specific task.Among them,the formation of multi-agent is an important issue for coordination control in multi-robot systems,which is based on path planning.The application of a complex function can be regarded as a special multi-agent collaborative path planning problem.Therefore,multi-agent collaborative path planning and multi-agent formation encirclement have gradually become a recent research hotspot.This paper is based on multi-agent reinforcement learning to solve the path planning problem and formation encircling problem in collaborative welding tasks.Different from the existing results,the algorithm design does not depend on an accurate system model,and at the same time it can solve the problem of multiple balance points caused by obstacle avoidance.Designing new reward function and loss function to train multi-agent depth deterministic strategy gradient algorithm(MADDPG).The design of multiple sets of control simulation experiments verifies the effectiveness of the algorithm proposed in this paper.This article mainly completed the following work:1.Aiming at the circular formation encircling problem of multi-agents considering collision avoidance,a reinforcement learning algorithm based on the MADDPG algorithm is proposed to enable the multi-agent system to control stationary or moving targets according to a circular orbit.Specifically,in order to achieve the dynamics of the goal,the formation of circular formation encirclement and obstacle avoidance,the reward function and its weight coefficient are designed based on encirclement,formation and collision avoidance.The simulation designed a number of scenarios,including the different trajectories of the target,and the target stationary and moving,to verify the effectiveness of the algorithm.2.The problem of multi-agent super-elliptical formation encircling is further studied.Compared to problem two,the expected distance in the formation-based reward function is no longer fixed,but a distance that varies with the parameters.For this reason,a number of scenarios were finally designed for simulation,including ordinary elliptical formation encircling and super-elliptical formation encircling,and the different trajectories of the target's static and moving and moving to verify the effectiveness of the algorithm.3.Aiming at the problem of multi-agent collaborative path planning for multi-robot collaborative welding,a reinforcement learning algorithm based on the MADDPG algorithm is proposed,so that each intelligent body in the multi-robot system can find a safe(obstacle avoidance)route in collaborative welding.Around the welding task,the reward function and its weight are designed based on distance optimization and collision avoidance.The simulation designed multiple scenarios,including the distribution of different obstacle positions,the number and length of different welds,whether the welds are tilted,and the dynamic models of different robots to verify the effectiveness of the algorithm.