Research Of Path Planning Algorithm For AUV Based On Reinforcement Learning

With the development of intelligent technology in AUV(Autonomous Underwater Vehicle),more and more methods of Artificial Intelligence are applied in its key technologies.Reinforcement learning is an important method of Artificial Intelligence,which learns decision models by itself in the interaction with the environment,thus weakening the constrains on model and environment and having a high degree of flexibility.Based on reinforcement learning algorithms,this paper focuses on long-distance 2D path planning problems of AUV under the influence of currents to improve the flexibility and adaptability of AUV in complex environments.The main research contents are as follows:Firstly,based on the value iteration network,this paper analyzes its problems and improves it using the gated idea and dueling architecture,thus proposing the improved value iteration network which is then applied in the global path planning of AUV to obtain a shortest global path.Secondly,in view of the shortcomings of DQN(Deep Q Network)which models the mean of action value,this paper uses neural networks to directly approximate the probability distribution of the action value.Besides,the double-Q learning technique is added in training process and the distributed DDQN algorithm is proposed.According to the special requirements of AUV obstacle avoidance in unknown environments and the influence of currents on AUV's movement,a local path planning algorithm based on the distributed DDQN is designed to obtain a local path with minimum time cost.At last,an AUV path planning system is designed by integrating the global and local path planning algorithms.A priori maps and detection information of sensors are effectively utilized that enables AUV not only to plan efficiently an optimal global path,but also to reduce time with the help of currents and respond to emergencies.In order to execute the planned path,a path tracking controller based on LOS(Line of Sight)navigation algorithm and PID controller is designed.In addition,a visual simulation platform is developed using the model of REMUS underactuated AUV as the simulation object.Simulations have verified the effectiveness of the AUV path planning system designed in this paper.