Research On Optimal Consensus Control Of Multi-agent System With Unknown Dynamics Based On Reinforcement Learning

Consensus control of multi-agent system has attracted attention of many researchers because of its wide application in the engineering field,and optimal consensus control has always been a hot research topic.In practical applications,due to the complexity and unknowability of the system,the dynamics of system may not be completely obtained.Therefore,the research on optimal consensus control with unknown dynamics models has important practical significance.Furthermore,considering the differences in the dynamics and interactions of the agents in the system,the optimal bipartite consensus control problem for heterogeneous systems is raised.In this paper,we utilize algebraic graph theory,Lyapunov stability theory,reinforcement learning and other methods to study the optimal consensus control and optimal bipartite consensus control of discrete-time multi-agent systems.The main research contents are as follows:1.The optimal consensus control problem of discrete-time multi-agent systems with unknown dynamics is studied.We propose a novel policy iterative algorithm by combining reinforcement learning method and control theory in this paper,which can overcome the problem of traditional algorithms relying on accurate system dynamics models.The proposed algorithm can learn optimal consensus control protocol by training the neural network,which only needs the error information of the agent's own state and neighbors' state.And the simulation experiment proves that compared with the traditional algorithm,the proposed algorithm can speed up the convergence,reduce the training volatility and be more stable.2.The problem of optimal consensus control for heterogeneous grouped discretetime multi-agent system with unknown dynamics is solved.Under the directed communication topology,a generalized model of grouped multi-agent system is established,in which the intragroup dynamics of agent is same but the intergroup dynamics of agent is different,and all agents dynamics are unknown.The proposed novel policy iterative algorithm is used to learn the optimal control protocol and achieve optimal consensus control.And the effectiveness of the algorithm is verified by simulation experiment.3.The optimal bipartite consensus control problem of heterogeneous discrete-time multi-agent systems with unknown dynamics models is discussed.The signed weighted digraph is introduced to represent the communication topology of the system.By combining the ideas of reinforcement learning and deep learning method,a deep ActorCritic(AC)neural network model is proposed to implement a policy iteration algorithm,which learn optimal bipartite consensus control protocol by training the deep AC network.And the effectiveness of the algorithm can be verified by the results of simulation experiment.