Research On Navigation Of Multi-autonomous Underwater Robot Based On Reinforcement Learning

The navigation problem is a necessary part to achieve the autonomy and intelligence of underwater robots.The navigation problem of the autonomous underwater vehicle requires the robot to acquire the underwater environment information first,and then according to the acquired environmental information,an optimal collision-free path from the starting point to the target point is planned to guide the underwater robot swimming.Traditional path planning algorithms such as fuzzy algorithms,neural networks,etc.generally require underwater robots to know their environment beforehand.However,under normal circumstances,the underwater robot operating environment is not known;On the other hand,the tasks required for underwater robots are complex and more and more robots need to cooperate with each other to complete the tasks.Under the dual factors,the traditional algorithms are more and more difficult to meet the requirements.The emergence of reinforcement learning makes it possible for underwater robots to explore the environment autonomously and learn an optimal path autonomously.On the basis of reinforcement learning,underwater robots do not need to know the environment where they are.By relying on their own exploration and learning of the environment,they can plan an optimal collision-free path from the starting point to the target point.Therefore,based on reinforcement learning,this thesis studies the path planning problem of multiple autonomous underwater vehicles.The main research work of this topic is discussed as follows:(1)A special underwater robot,that is bionic robot fish,is studied,and a new structural design method is proposed.According to the size data of the real fish body,the contour curve of the robot fish is fitted combining with the least squares method.Then a single joint robotic fish is designed.(2)Based on the Q learning method in reinforcement learning,the simulated annealing algorithm is used to optimize the "exploration and exploitation" balance problem,and the path planning of a single robot fish is studied.A reward function based on the goal-oriented idea is proposed,and the stopping condition of the algorithm is optimized to form an improved simulated annealing-Q learning algorithm.Finally,the simulation analysis of a single robot fish is carried out in a static environment and a dynamic environment.(3)On the basis of the previous work,the path planning problem of multiple bionic robot fish is studied.The simulated annealing-Q learning algorithm which is suitable for multiple robotic fish path planning problems is optimized from the two aspects of action strategy and reward function.Taking two robotic fish and three robotic fish as examples,the simulation analysis is finished in 10?10 and 20?20 environments respectively.(4)Prototype of the designed bionic robot fish is produced and debugged,and static experimental analysis is carried out to verify its balance and sealing,and then the linear fish swimming experiment analysis of the robot fish is conducted.Finally,based on the improved annealing-Q learning algorithm,the path planning experimental problems of single and multiple bionic robotic fish are analyzed.