Research On Multi-AUV Task Assignment And Path Planning Based On Reinforcement Learning

With the increasing depletion of land resources,major maritime powers have shifted their research focus to the vast domain of ocean exploration.Consequently,research on Multiple Autonomous Underwater Vehicle(MAUV)technology has received increasing attention.The complex ocean environment places high demands on the cooperative operation of multiple AUVs,and a reasonable task allocation method can significantly improve the task capability of multiple AUV systems.Meanwhile,reasonable planning of task paths is an important prerequisite for the completion of tasks by multiple AUVs.Therefore,the task allocation and path planning technology of multiple AUVs have significant research significance.This paper mainly focuses on the task allocation problem of multiple AUV systems and the path planning problem of AUVs,with the following main research contents:(1)This paper analyzes and establishes a six-degree-of-freedom kinematic model of AUVs;and estimates the ocean current model in the real environment by superimposing multiple Lamb eddy current fields,which provides an ocean current model for the path planning algorithm in later sections,enhancing the realism and robustness of the path planning algorithm.In addition,this paper analyzes and studies several classic algorithms in reinforcement learning,which provides a theoretical basis for the design of task allocation and path planning algorithms in later sections.(2)This paper proposes a dynamic Q-information pheromone ant colony algorithm to solve the problem of task allocation in multiple AUVs with limited resources.Based on the MTSP model,a task allocation problem model of multiple AUVs with limited resources is established.The MTSP model is simplified by adding virtual nodes for solution.Meanwhile,the ant colony algorithm parameters are reasonably selected through experimental analysis,and the problem of early stagnation in the ant colony algorithm is solved by introducing the dynamic Q information pheromone update method and the variable epsilon-greedy path selection strategy.Furthermore,the problem of task allocation in multiple AUVs with limited resources is effectively solved.(3)This paper establishes a custom reinforcement learning environment and trains the AUV model using the PPO algorithm,successfully solving the path planning problem of AUVs in an unknown environment.A sonar detection model is established,and the state-action space and reward function are reasonably set,with the introduction of ocean current interference,completing the construction of a custom AUV reinforcement learning environment.Through training,an AUV path planning model is obtained,and the feasibility of reinforcement learning in solving the AUV path planning problem is verified through the effectiveness of AUV path planning in different scenarios.(4)The USV water experimental platform is built to verify the task allocation algorithm of multiple robots with limited resources.Through the task allocation algorithm,tasks are assigned to the USV,and the quality ·of the USV’s task path is analyzed to observe the performance of the task allocation algorithm in a real environment,verifying the effectiveness of the task allocation algorithm.