Research And Implementation Of Multi-USV Formation Path Planning System Based On Deep Reinforcement Learning

Unmanned Surface Vehicle(USV)is an important member of the marine robotics family and can be widely used for tasks such as marine environmental observation,territorial sea monitoring and maritime search and rescue.As the capabilities of a single USV are largely limited,the formation of multi-USV formation is an effective way to extend the capabilities of a single USV in order to meet a variety of complex tasks on the sea.Multi-USV formation can perform more complex tasks and improve the robustness and fault tolerance of formation systems.Traditional methods have limitations with complex tasks in Multi-USV formation,and the rapid development of deep reinforcement learning can ensure that USV make intelligent action decisions in dynamic environments.Therefore,this thesis takes USV as the experimental object to study the problem of intelligent formation of unmanned ships.The main research elements of this thesis are as follows.By analyzing the current situation,research background and significance of multi-USV formation control at home and abroad,and focusing on the limitations of the current multi-USV formation control,a deep reinforcement learning-based multi-USV formation strategy model is proposed.The LOS algorithm is combined in the formation strategy and the proposed formation strategy has a strong migration potential to other multi-agent systems due to the neglect of the dynamics model of the USV.In order to evaluate the performance of the formation strategy,two different scenarios were designed based on the practical tasks performed by the multi-USV system,including observation aperture enhancement with the desired formation and dynamic non-cooperative target roundup.In addition,the proposed multi-agent deep deterministic policy gradient(MADDPG)algorithm is optimized to address the problem of slow convergence during model training due to inexperience.The incorporation of a prioritized experience replay method enables the model training process to converge more quickly and perform more robustly.The performance of this formation strategy has been verified in both simulated and real environments.Compared with other deep reinforcement learning algorithms as well as traditional methods,the proposed MADDPG-based formation strategy can improve formation task success,formation stability and formation generation capabilities,and has utility in the surface environment.Based on the application requirements of unmanned ship formation strategy,a deep reinforcement learning-based path planning system for multi-USV formation is designed.The various system functional modules are designed in detail for the issue of multi-USV formation control through a requirements analysis of the system.The system includes a path planning module,a model training module,a result management module and a model testing module,which facilitates the construction and testing of multi-USV formation strategy models and effectively improves the efficiency of multi-USV formation path planning.After testing the functional and non-functional aspects of the multi-USV formation control system,the test results show that the system can operate stably and meet the needs of the multi-USV formation.