Research On Convention Emergence Method Based On Multi-player Synchronous Interaction Model

Many systems in real life can be regarded as Multi-Agent Systems,these systems are composed of individuals who can perceive the environment and perform corresponding responses to the environment according to the perceived information.These individuals are called agents in the research area of artificial intelligence and computational social science.Many systems in real life can also be view as complex networks,the individuals in the system is view as the nodes in a network,the edges of the network are determined by whether the individuals in the system are connected.Combining both the research methods of multi-agent system and complex network,introduced a new method to model the reality systems-networked multi-agent system.That is to say,the network topology of the system and the intelligence of the individuals in the system are considered at the same time.Reducing the conflicts among agents in the system will help to promote the coordination of the system and make the whole system work more efficiently.Previous researches found that it is an effective mechanism to make the agents adopt a consistent behavior to solve the problem of coordination in reality systems,these consistent behaviors are named as conventions by researchers,and the process of the formation of these consistent behaviors is called convention emergence.On the macro level,there are two main mechanisms to promote the emergence of convention.The first one is the top-down manner,this manner is to manage and regulate the behavior of agents in the system through a central manager,and hence make the agents adopt a consistent behavior.The second one is the bottom-up manner,in this way,the agents in the system interact with each other continuously,and adjust their behaviors gradually according to the interaction information,finally,form a consistent behavior.The top-down mechanism often lacks robustness,the behavior of agents in the system cannot be adjusted when the manager agent fails,or the regulatory information may not be able to transmit to all the agents in the system due to the environmental factors even if the manager agent works.To take advantage of the agents' ability of perceiving the environment and perform corresponding responses to the environment,the bottom-up manner often have a more robust performance.The robustness of this mechanism comes from two aspects,firstly,every agent in the system is homogeneous and have same influence on the system,it will not cause the whole system to fail due to the failure of one particular agent,secondly,agents do not need to obtain the environment information of the whole system,they can adjust their behavior merely according to their interaction information.In the research area of the bottom-up mechanism,One of the main research directions is to investigate how agents can effectively adjust their behavior according to their interaction information,and then facilitate the emergence of convention in system.This thesis studies the problem of convention emergence in the networked multi-agent system,the research scenario is that,in the networked multi-agent systems,agents interact with their neighbors through the multi-player synchronous interaction model,the agents are not informed the actions played by their neighbors,and the only information that agents can perceive is whether an interaction is success or not.In such scenario,this thesis analyzes two limitations of exiting methods.Firstly,non-learning methods rely on the observation ability of agents,agents with those methods may adjust their actions blindly when they can not inform the actions played by their neighbors,this trouble will become more serious when the agents have a lot of available actions to choose.Secondly,system is likely to form local subconvention in the system with isolated regions when agents use the learning algorithms such as Q-learning,once the subconvention emergences,learning algorithm has the positive feedback characteristic in the isolated regions due to the differences of interaction frequency between regions,such kind of positive feedback characteristic makes agents usually fail to coordinate their actions with each other,and then the whole system may fail to form global convention.In view of the above two problems,this thesis proposes a more effective action selection algorithm to facilitate the emergence of convention namely Win-Stay-Lose-Learn(WSLL).WSLL divides one agent's state into Win and Lose according to whether the agent's action is consistent with most of its neighbors,the state of Win means that the action of agent is consistent with most of its neighbors conversely,we say the state of Lose means that the action of agent is not consistent with most of its neighbors.Agent regulates its behavior with different strategies according to different states.The agent uses the Bellman equation to learn the expected benefit of each action and adjusts its action according to the expected benefit in the state of Lose.Otherwise,the agent keeps its current action and reset its learning experience to prevent the emergence of subconvention in the state of Win.To verify the performance of WSLL algorithm,this thesis compares WSLL with some existing methods under different experimental conditions.The experimental results show that WSLL is more robust and effective than the comparison algorithms.