Multiple AUV Cooperative Path Based On Reinforcement Learning Research On Planning Algorithm

Path planning,as an important research direction of AUV,aims to explore the optimal navigation route from the starting point to the target point based on underwater environment data and according to mission requirements and relevant criteria.With the development of AUV technology and the continuous improvement of task requirements,in most cases,a single AUV is difficult to meet the task requirements,and multiple AUVs need to work together to improve the overall task efficiency.Therefore,multi-AUV collaborative path planning research has become particularly important.The conventional path planning algorithm is highly dependent on the environmental model,and some problems such as local convergence and unstable planning results may occur in the experiment and practical application.In addition,multi-constraint conditions and cooperative relations among AUVs in complex environment need to be considered in the study of multiAUV collaborative path planning.Based on the above description,based on reinforcement learning thought,in real environment model is established on the basis of the Marine environment data,the result of the AUV global path planning,path smoothing method,multiple AUVs collaborative path allocation method,multiple AUVs coordination problems such as path planning method to research,implement the AUV more cooperative path planning algorithm based on reinforcement learning research purposes.Firstly,the principle of reinforcement learning algorithm and its improvement measures are studied.The relevant theoretical knowledge is introduced,and several commonly used reinforcement learning methods are analyzed and compared.Combined with the characteristics of the research object in this paper,a typical Q learning algorithm is selected as the basis of this study,and its feasibility is verified through experiments.Secondly,the improvement measures of environment modeling method and Q learning algorithm are studied.The common environment modeling methods are discussed,and 3d environment modeling is carried out by using existing Marine environment data.On this basis,the single AUV path planning task is completed.Aiming at the slow iterative speed of Q learning,a global path planning method of AUV based on improved Q learning is designed using the idea of hybrid search strategy.In order to make the path results conform to the motion characteristics of AUV,the non-uniform rational B-spline algorithm is used to realize the smooth calculation of 3d path results,and its superiority is verified by experiments.Then,the multi-AUV collaborative path allocation problem is studied.In order to achieve the best mission efficiency,based on the analysis of multi-AUV cooperative constraints,a path allocation model for multi-AUV cooperative navigation task is established.An improved swarm intelligence optimization algorithm is used to solve the optimal allocation strategy of the path allocation model,which solves the path allocation problem when AUVs work cooperatively,and its effectiveness is verified by experiments.Finally,the multi-AUV cooperative path planning problem with complex constraints is studied.To satisfy multiple AUVs in the task space and time in the process of collaborative,based on reinforcement learning theory,the thought of fusion level decomposition,multiple AUVs cooperative path planning problem is decomposed into path planning and collaborative planning layer respectively research and calculation,and through the experiment could satisfy the requirement of time,space,the AUV more optimal path results together.