Study On Optimal Consensus Of Multi-agent Systems Based On Q-learning

Inspired by the phenomenon of natural biological clusters,multi-agent systems are proposed and the bio-cluster phenomenon is extended to scientific research.Compared to the single agent,multi-agent systems can perform more complex and dangerous tasks in a more efficient manner.Due to its theoretical research significance and wide-ranging applied background,cooperative control of multi-agent systems is still one of the key research directions.Consensus control is one of the basic problems of cooperative control for multi-agent systems.Because it is difficult to obtain the accurate system model in many practical applications,it is necessary to propose a model-free control method to achieve the optimal consensus of the multi-agent systems.Based on the model-free Q-learning method,this paper studies the optimal consensus control of a series of multi-agent systems.The main research contents are as follows:Firstly,the application of Q-learning method in discrete-time linear single-agent system is realized.The feasibility of Q-learning method is verified.The consensus control of single-agent system is the general tracking problem.In order to realize the optimal control of the system without knowing the accurate system model,a model neural network is designed to identify the system.Then based on adaptive dynamic programming and Q-learning,the modeless controller is designed.The optimal control is obtained by reinforcement learning and least square method.The effectiveness of the controller is verified comparatively by simulation examples.Secondly,Q-learning is applied to the multi-agent system,where the Q-learning method is extended to the consensus control problem for multi-agent systems from the tracking control problem for single agent systems.Considering that the system information is not easy or impossible to be obtained in real-world applications,the data-based Q-learning method is adopted for a class of discrete-time multi-agent systems,which are known as graphic game.Reinforcement learning and least square method are used to obtain the analytical solutions of optimal control.Finally,the simulation examples prove the effectiveness of the proposed algorithm.Finally,the implementation of Q-learning method in heterogeneous multi-agent system is studied.Considering the uncertainty of information exchange between the agents,the Q-learning algorithm based on policy iteration is proposed.Firstly,the observers for estimating the leader's dynamics are designed and trained under the switching topologies.Then the trained observers are applied to the Q-learning algorithm.The effectiveness of the algorithm is illustrated by numerical simulations.