Reinforcement Learning Of Motion Control Strategy In The Environment Of Multiple Mobile Robots

The system of multiple mobile robots is one of the main research content in robotics. Path planning and motion control is the key for mobile robots to accomplish their tasks. Path planning and motion control by machine learning (especially reinforcement learning) in multi-robot environment is one of the centers of attention in robotic research. In this thesis, the reinforcement learning of motion control strategy is studied for the environment of multiple mobile robots. In this thesis, the research is carried out in the following four aspects: environment modelization and planning in dynamic environments, path planning by learning and optimization in multi-obstacle environments, the improvement of reinforcement learning algorithm, and reinforcement learmng of control strategy based on the simulation experimental platform of multi-robot system.An environment modelization method is proposed based on the fuzzy concept and the theory of possibility. Based on the proposed modelization method, dynamic motion control is accomplished by fuzzy decision, which achieves real-time obstacle-avoiding planning results.A path planning method is proposed by incorporating the artificial potential field into the ant colony optimization. The artificial potential field is introduced as the priori knowledge to initialize the solution of the ant colony optimization. Compared with the current method of path planning by optimization, the proposed method notably improves the planning efficiency.One of the key problems in traditional reinforcement learning algorithm is that the learning efficiency is low in large state space. To solve the problem and improve learning efficiency, the fuzzy-state reinforcement learning algorithm is proposed. The scale of learning in discrete state space is studied, based on which a multi-scale learning algorithm is proposed. Moreover, the ant colony optimization is incorporated into reinforcement learning, based on which the reinforcement ant learning algorithm and the delayed-optimization learning algorithm are proposed.The proposed learning algorithms are applied to the path planning in multi-obstacle environments. Compared with the planning method by traditional reinforcement learning, the proposed methods notably improve the planning efficiency.The uncertainty of observed state and the uncertainty of the policy in learning are studied. Based on the concept of random variable's entropy, the policy entropy is proposed for reinforcement learning as the measurement of the policy's uncertainty in learning (i.e. the convergence degree of the policy). The state entropy is proposed for reinforcement learning as the measurement of the state's uncertainty caused by the incompletion of the observed information. A reinforcement learning algorithm with self-adaptive learning rate is proposed based on the policy entropy. The proposed algorithm is applied to the path planning hi multi-obstacle environments. Compared with the planning method by traditional reinforcement learning, the proposed methods notably improve the planning efficiency.The improved reinforcement learning algorithm is applied to the learning of multi-robot control policy based on the "TeamBots" multi-robot simulation experimental platform. The policy learning is successfully achieved for the multi-robot foraging and robot soccer task, hi the experiments of soccer robot, the cooperation and task allocation is achieved based on the proposed learning algorithm. Compared with the traditional reinforcement learning method, the proposed learning method improves the planning efficiency.