Multi-agent Reinforcement Learning Model That Forms Individual Division Of Labor Under Social Dilemma

A single agent using deep reinforcement learning has been able to solve decisionmaking problems such as complex board games.However,many real-life decisionmaking problems require the cooperation of multiple agents to complete.Such multiagent decision-making problems often have income conflicts between individuals and groups.Designing a multi-agent decision algorithm that can resolve such conflicts is one of the hot research issues in deep reinforcement learning.In the group decision-making process,individuals within the group often form a specific division of labor.However,it is unclear how the individuals within the group form a mechanism of division of labor.Therefore,in this paper,based on deep reinforcement learning,when agents are in a certain type of conflict-social dilemma,individual agents in the group form a specific division of labor influencing factors,and then propose a multi-agent reinforcement learning algorithm based on satisfaction.The main work and results of this article are as follows:1.Designed a decision task with the characteristics of social dilemma.The decision task simulates the dynamic change process of natural resources and waste in human society.The agent needs to make a trade-off between collecting resources and recycling waste in this decision-making task.If the agent continues to collect resources and ignores the recycling of waste,the growth of waste will occupy the growth space of the resource,which will limit the overall income of the agent;while the agent's continuous recovery of waste will impair its individual income,but other tasks Agents will benefit.Decision-making tasks with such trade-offs exhibit characteristics similar to social dilemmas,which can be regarded as a type of "prisoner's dilemma." By designing an agent's deep reinforcement learning algorithm,it is verified that this decision task can simulate the real social dilemma of conflicts of interest between individuals and groups.2.A multi-agent division and cooperation strategy algorithm based on satisfaction degree is proposed.Based on the strategy algorithm of deep reinforcement learning,it is proposed that each individual in multi-intelligence needs to introduce the measure of satisfaction,so that the strategy of the agent can be designed in a decision task space to balance the conflicts of interest between the individual and the group.In particular,when there is heterogeneity in satisfaction between agents,a significant division of labor can be formed between agents.Simulation experiments also show that the size of the agent's field of view and different initial positions will also affect the formation of the agent's division of labor.Based on the realistic social dilemma environment,this study designs a decision task with conflicting interests between individuals and groups,and proposes a multiagent deep reinforcement learning strategy algorithm based on satisfaction degree to solve the decision task.The future research direction is to apply the algorithm to practical engineering applications such as collaborative cooperation of drones.