Research On A Type Of Multi-Agent Cooperative Control Problem Under Multiple Decision-Making Mechanisms

As the constant development of networked systems and intelligent control theory,cooperative control of multi-agent systems has shown broad application aspects in a lot of fields such as multi-robot systems and wireless sensor networks.Meanwhile,the development of information technology further leads to the increase of system complexity,demand diversity,as well as intelligent level of a single individual,which make the individuals usually weigh and balance their own interests when making decisions.Herein,we consider this type of multi-agent systems with self-interest agents.Obviously,the self-interest behaviors of individuals cannot always have a positive impact on realizing the system goal.Therefore,studying how to balance the individual interest and the system goal has become an important topic in the field of multi-agent cooperative control.Apart from the individual self-interest behaviors,the communication graph is another important issue that influences the cooperative behaviors of a group of agents.Under the different structures of the communication graph,self-interest individuals show different interaction relationship and different decision-making behaviors.Game theory is a powerful tool to characterize and analyze the interaction mechanism in selfmotivated multi-agent systems.However,existing works usually focus on simple interaction environments,and characterize the interaction mechanisms among a small number of agents with equal positions.There is a lack of deep and systematic study on complex interaction scenarios.In view of such,we further consider the interaction and decision-making mechanisms with different communication structures,and investigate multi-agent cooperative control problem under several types of decision-making mechanisms.We first focus on the case of simultaneous decision making,in which all the agents are in equal positions and make their decisions simultaneously without knowing others’ responses.Then,considering the individuals that can make decisions prior to others,the cooperative control problem under sequential decision-making mechanisms is studied with the asymmetric interaction among individuals.Finally,we pay attention to the cooperative control of large-scale systems with more complex decision-making mechanisms.To deal with the above problems,we propose several game-theoretic based methods to characterize different interaction mechanisms and develop some reinforcement learning based approaches to approximate the equilibrium strategies.The main works and contributions are summarized below.1)To deal with the nonlinearity induced by input saturation constraints under a simultaneous decision-making mechanism as well as the difficulties of designing cooperative controllers with no prior information on system dynamics,we integrate the coordination goals with the individual constraints on input saturation to design the excitation functions.Then,based on the framework of non-cooperative games,we build the connection between the cooperative controllers and the equilibrium strategies.Furthermore,we propose a model-free off-policy reinforcement learning algorithm to approximate the cooperative controllers,which provides a feasible way to realize cooperative control for input-constrained multiagent systems without any information of the system dynamics.In addition,we remove the requirement of model-free control algorithms on the probing noise,and develop a primal-dual optimization algorithm for the design of the modelfree controllers.This algorithm works well in high-dimensional systems and also provides an interesting explanation for the classic policy iteration algorithm from a primal-dual optimization perspective.2)To overcome the challenges caused by the asymmetric interaction in multi-agent systems as well as the limitation that the reinforcement learning algorithms for seeking equilibrium strategies depend upon stable initial strategies,we first investigate the sequential decision-making problem of multi-input dynamical systems,and then introduce the sequential decision-making mechanism into the selfmotivated multi-agent cooperative control problems.We design an excitation function for each agent with the consideration of the individual roles and the system goals,propose a unified framework to characterize the simultaneous and the hierarchical decision-making mechanisms,and analyze the uniqueness and stability of the equilibrium strategies.Furthermore,we develop the reinforcement learning algorithms,with the abilities of ensuring the stability of iterative strategies,to seek the equilibrium strategies,which successfully solve the hierarchical synchronization control problem of multi-agent systems with partial information of system model or without any model information.3)In the large-scale systems,the individuals have variable goals and the interaction mechanisms between the individual and its neighbors are unclear,which make the analysis and design of cooperative controllers extremely difficult.To solve the dynamic collective choice problem of large-scale systems,we employ the mean-field games to make the interaction between the individuals and the group approximately equal to the complex interaction between the individual and its neighbors based on mean-field games.We design non-convex and non-smooth individual excitation functions to characterize the individual goals,and further design and analyze the decentralized equilibrium strategies by using the abstract fixed point theorem,which overcome the challenges resulting from the imbalance between the heterogeneous individual goals and the system goals in large-scale systems.The complexity of the proposed methods is independent of the number of agents.Furthermore,the designed controllers could reach the Nash equilibrium of the mean-field game when the number of agents approaches infinity.