Research On The Distributed Control System Of Multi AUV Based On Reinforcement Learning

In recent years,autonomous underwater vehicle(AUV)plays an increasingly important role in marine environment data collection,underwater archaeology,resource exploration and other fields.As the working environment is more complex and changeable,and the requirements for work efficiency are improved,the single AUV has been unable to perform some tasks,so the multi-agent formation mode has a prominent advantage.Multi AUV system can be distributed in the same space according to the planned formation to cooperate with each other when performing tasks,and complete the specified tasks by carrying sensors or manipulators.Therefore,formation has the advantages that a single robot can't reach,and the motion control of formation has become a major research focus.Based on an artificial intelligence project,aiming at the cooperative control problem of multi AUV formation,this paper attempts to use a distributed control method combined with reinforcement learning to improve the learning ability of formation system and achieve accurate underwater motion control effect.Firstly,a model free distributed control method based on reinforcement learning algorithm is proposed.In this part,the principle and improvement method of actor critical algorithm are described firstly,and then a model-based distributed control system is designed by combining the algorithm with algebraic graph theory,At the same time,the design elements of reinforcement learning algorithm are described.The simulation results show that the designed distributed control system can basically ensure the AUV formation to track the path,and keep the formation change error within a certain range.Secondly,a parameter adaptive distributed control method based on actor critical algorithm is proposed.In order to solve the problem that the distributed control system designed in the previous part is not fast and stable enough,and that most of the non intelligent distributed controllers do not have learning performance,and parameter adjustment strictly depends on prior experience,this section combines actor critical algorithm with distributed control method based on backstepping,and designs a new parameter adaptive distributed controller.Through the simulation experiment,it is proved that this control method optimizes the performance of the controller by adjusting the parameters on the basis of the traditional backstepping distributed controller,so that the AUV formation trajectory tracking error is smaller and the formation keeping ability is better.Finally,based on the above work,a distributed unknown disturbance observer is designed.In this part,the feed-forward observation system is added to solve the problem that the motion environment of multi AUV formation is complex,the current interference is strong,the feedback control system alone is not enough to offset the interference,and the formation is prone to distortion.The simulation results show that the compound control system composed of feedforward observer and feedback controller has excellent control effect on formation tracking and formation keeping under strong disturbance.Considering the feasibility of the above control algorithm in practical engineering,a simulation system combining software and hardware is designed.Through the combination of embedded development board and graphical simulation interface,the communication principle of each part in the actual project is simulated,and the feasibility of the distributed control system in the actual project is proved.