Research On Multi-Robot Coopertion Mechanism Based On Reinforcement Learning

With the development of robotics, the capability of a robot is becoming more powerful. The application of robotics has extended into many domains, from automatic assembly to deep sea exploration, mars exploration and so on. There are some specialized tasks which single robot has not enough power to complete, but multiple robots can be organized to complete. It is very difficult for the designer to provide proper control parameters which can be used to help the multi-robot system consisting of many autonomous robots work cooperatively. Therefore, it is highly desirable for multi-robot system and each robot to be able to learn control parameter values in order to optimize their task performance, and to adapt to dynamic environment.Reinforcement learning, a kind of machine learning algorithm, has receieved much attention in recent ten years. Reinforcement learning does not need priori knowledge and improves its behavior policy with knowledge obtained by interaction with the environment. So reinforcement learning has the ability of self-learning.Reinforcement learning has been applied to single agent environment successfully. Due to the theoretical limitation that it assumes that an environment is Markovian, traditional reinforcement learning algorithms cannot be applied directly to multi-agent system. In this paper, two-layer reinforcement learning method for multi-agent cooperation is presented. This method is realized by adding two-layer reinforcement learning unit to every agent. The first layer is for learning globalcooperation game, and the second layer is for learning efficient action game in its own view. An experiment that three agents raise a round object by cooperation is made to test the efficacy of the method. The experiment results show that the agents using this method cooperate better than the agents using center reinforcement learning.In multi-agent system based on reinforcement learning, the evaluation to the behavior of a robot depends on the other agents' behaviors closely. If each agent takes its action after it predicts the other agents' actions self-consciously, the performance of the whole system will be better. In this paper, introducing joint-action to the traditional reinforcement learning, a new multi-agent reinforcement learning algorithm based on behavior prediction is presented and several methods for predicting other agents' behaviors are discussed. The experiment results show that joint-action is applied in the traditional reinforcement learning successfully and the cooperation process can be speeded by behavior prediction.