Deep Reinforcement Learning For Robotic Cooperation

As a new star in the field of artificial intelligence,reinforcement learning has successfully solved sequential decision-making.As a future development trend,combining with robots,it is conducive to manipulation tasks in unstructured environments.With the complexity of actual tasks,it is urgent to break through the limitations of a single robot's capabilities.Therefore,intelligent multi-robot cooperation emerges at the historic moment.Based on multi-agent reinforcement learning,this paper proposes an algorithm for multirobot collaboration in response to the difficulty of exploration and the lack of effective data.Taking agents with collaboration strategies as the research object,the aim is to achieve robot cooperation and autonomous gripping,in the framework of multi-agent reinforcement learning and imitation learning,and to conduct in-depth discussions from the level of algorithm optimization.The main content is divided into the following aspects:(1)The problem of a large amount of invalid data in the early stage of reinforcement learning algorithm training is discussed,and hindsight experience replay technology(M-HER)based on multi-agent learning is proposed.Replacing random data in the experience buffer with virtual targets solves the multi-goal situation under sparse rewards,helps the robot acquire a large number of effective operation skills in the early stage,and is conducive to strategy updates.(2)This paper studies high-dimensional continuous space in robots,and a collaborative strategy based on imitation learning(RTLf D)is proposed.While driving the robot to explore itself,it uses a small amount of expert data to guide,the combination of them is not constrained by expert data,and efficiently solve the difficulty of multi-agent learning from scratch.Aiming at the errors and imperfect sequences in the actual collection of expert data,a Judge mechanism is proposed to distinguish the pros and cons of expert data and obtain a stable strategy.The simulation verifies the effectiveness of the algorithm and shows that imitating collaboration and Judge mechanisms can help robots form efficient strategies.(3)Based on the reinforcement learning framework of the imitation learning mechanism,a multi-agent imitation learning algorithm based on the reward-driven generative adversarial network is proposed.The on policy method is used as the core of the interaction between the robot and the environment,and a curiosity module with two sub-structures is proposed.Based on intrinsic rewards,the robot is driven to explore more new states and actions,and the optimal strategy is searched for on the basis of expert strategies.To prevent excessive exploration,three methods(parameter sharing,value cutoff,and time cutoff)are proposed to constrain the robot's exploration range and improve actual performance.(4)The characteristics of robot collaboration are analyzed,a physical simulation experiment environment is built,and the observation and action space in the framework is defined.Developed an interface based on the algorithm environment for information interaction.Discuss the difference between the simulation model and the real robot.Using the domain randomization,the anti-interference model is obtained and used for the real robot operation.In summary,this thesis takes multi-agent reinforcement learning as the basic framework,and aims at the robot team,starting from three aspects: increasing data utilization,accelerating training speed,and improving exploration ability,then training robots to form a collaborative strategy.