Application And Simulation Of Multi-Agent Q-Learning Algorithm In Multi-AUV Cooperation

Autonomous Underwater Vehicle (AUV) plays an important role in marine environment monitoring, seabed resources investigating, science inspecting, dangerous environment exploring, rescue and salvage and so on. Along with the mission become more complicatedly, the single AUV obviously shows insufficient at effectiveness, robustness and flexibility in a large-scale operating mission. It is necessary to coordinate with the help of Multi-AUVs operating together, and cooperation is the key techniques of Multi-AUVs . In this paper a cooperative strategy, a Multi-AUVs cooperative system's design and it's simulation are discussed.Firstly, the trend of multiple AUVs development and the research meaning are presented. Then a AUV dynamic model, multiagent Q-learning model is designed based on the need of multiple AUVs system, and a new architecture of AUV based on mission is proposed. Then, several single agent and multiagent reinforcement learning algorithms proposed in recent years are investigated, compared and analyzed deeply in this paper. And a muliagent Q-learning algorithm is proposed. This algorithm involves simple procedures and easy computations, and can guarantee good learning convergence. Experiment results of multi-AUV's coordination and control show that this algorithm is effective.Then, a Multi-AUVs cooperative system based on muliagent Q-learning algorithm is desinged. Several simulation experiment show the application of muliagent Q-learning algorithm in multi-AUV's coordination. And the result show this algorithm can converge to Nash equilibria, avoids resolving complex multiple Nash equilibria problem, and it is effective and converges well.