Research On Evolution Of Cooperative Behavior On Complex Networks

Cooperative behavior exists widely in human society and nature,and plays a vital role in the maintenance and development of every life system.Understanding how cooperative behavior emerges and sustains in a population of selfish individuals remains one of the major challenges facing evolutionary biology and social sciences.The evolutionary game theory on complex networks provides a basic framework for studying the evolution of individual strategic competition and cooperative behavior.As a core part of the field of artificial intelligence,machine learning has given new development momentum in the directions of image recognition,semantic analysis,and game competition.Reinforcement learning is a branch of machine learning inspired by behaviorist psychology.It focuses on how the agent should act in an unfamiliar environment to maximize its cumulative reward.From this perspective,reinforcement learning algorithms are very suitable for the research of evolutionary game theory.Based on this analytical framework,this article focuses on the Monte Carlo method and uses computer simulation to explore the impact of reinforcement learning algorithms on the generation,maintenance and development of cooperative behavior.Firstly,in the context of the evolutionary prisoner's dilemma game,Q-learning,a classic algorithm in reinforcement learning,is introduced into the process of individual selection of learning objects in the game.Compared with the traditional random selection method,the difference is that the selection of learning objects under the new mechanism will completely depend on the information in the Q-table.That is,each individual will select the action with the highest Q-value(e.g.neighbor label)in the current state s with a certain probability.Through the results of numerical simulation,it can be found that the introduction of Q-learning algorithm can effectively promote cooperation regardless of the structure of the network topology.In particular,compared with the small and discrete clusters of cooperators formed under traditional circumstances,the Q-learning selection mechanism is more helpful for cooperators to gather into a large and tight cluster.In addition,we have observed that this profit-driven Q-learning algorithm can guide participants to obtain higher returns,and those cooperators with higher returns are easier to be imitated and learned.Finally,by analyzing the differences in the types of learning objects selected by game individuals with different strategies,it is found that under the traditional circumstances,individuals are more inclined to not change their strategies,while under the Q-learning selection mechanism,no matter what their original strategies are,the possibility of them learning from cooperators is greatly increased.Secondly,in the context of the evolutionary prisoner's dilemma game,we have introduced the concept of "agents".The agent's strategy update will be directly completed through the Q-learning algorithm.In order to explore the influence of these individuals on the evolution of cooperative behavior,we have introduced a proportion of agents(?)on the network,and the rest is a proportion of traditional individuals(1-?)who use Fermi update rule for strategy selection.The simulation results found that under the mixed linkage of these two kinds of people,the introduction of agents can significantly promote cooperation.And there is a very interesting phenomenon: there will be a special agent ratio ?(?=0.7),so that under different dilemma strengths,the cooperation rate on the entire network can reach a peak at this point.Based on these conclusions,we confirmed that agents can enhance the ability of cooperators to aggregate into clusters by influencing the strategic choices of traditional individuals in the surrounding neighbors.Moreover,the strategic responses of the two groups of people are also different: agents are more inclined to become cooperators based on their own judgments,while traditional individuals are more willing to respond to opponents' betrayal.These are the reasons why the level of cooperation has been improved.Next,we found that when 70% of agents and 30% of traditional individuals are randomly distributed on the network,under the interaction,the average payoff of each other can be increased,and the cooperation of surrounding neighbors is promoted.The cooperation rate has also reached the optimal level.Finally,we compare the results of single-state and multi-state environments,confirming that different state spaces also have a certain impact on the evolution of cooperation.This article provides a new development idea for the research of evolutionary games: to integrate the reinforcement learning algorithm with the related problems of evolutionary games.The combination of these two fields has enriched the existing theoretical results.It can provide a new thinking framework for solving economic,trade,environmental,resource allocation and other issues,and help us solve various social dilemmas related problems in real life.Of course,these contents also have certain reference significance in the spread and prevention of diseases,traffic management and control,and network planning and security.