On The Coadaptation Of The Swarm Robotic System

With the development of the society, it's required that a large scale of robots cooperate to complete the assigned missions in complex environment. The motivation for the research of multi-robot system based on that distributed multi-robot system has many advantages over the traditional complex single robot system, such as reliability, adaptation, maintainability and flexibility.The complexity and communication would expand exponentially as the scale of multi-robot system increases, and it can't be resolved by the traditional multi-agent theory. So it gives birth to swarm robotics. Swarm robotics researches on how a large scale of relative simple robots can interact locally to create complex collective behaviors, and is a novel approach to coordinate a large numbers of robots. It is inspired from the observation of social insects– ants, termites, wasps and bees– which can create collective intelligent behaviors by simple interaction and complete the complex missions that can't be completed by single individual.Swarm robotic system can be applied in domains such as the cleanup of toxic waste, planetary exploration, search and rescue missions, surveillance, and cooperative transportation. And it can forecast that the application of swarm robotic system will change the society greatly, and improve the standard of living and the modernization of national defense largely.The coadaptation of the swarm robotic system means that how robots or agents optimize their control policies and adjust their behaviors to adapt to the dynamic environment or tasks by means of interaction between itself and environment or other agent. The research of coadaptation has important influence on the swarm robotic system, and has been applied in many complex missions. This paper researches the coadaptation of swarm robotic system in some aspects as follows. First it studies the distributed coordination method based on market mechanism to allocate tasks in swarm robotic system, and then researches on the layered reinforcement learning to learn the obstacle avoidance policy. A cooperative reinforcement learning method based on the maturity of policy is then proposed to accelerate the learning. It also introduces some familiar self-organized collective pheromones in biological colony and the application in swarm robotic system. Last, the mathematical modeling method is applied to analyze the swarm robotic system. The details are as follows:(1) Research on the distributed cooperation method based on the market mechanism in distributed task-allocation of swarm robotic system. That is how to allocate tasks to robots in swarm robotic system without centralized control and meet optimization and reliability at the same time.Considered that the swarm robotic system is completely distributed, a distributed cooperative task-allocation method based on the market mechanism is designed, which takes the time required to complete the task and solving the deadlock problem as the criterion. And it includes the task-allocation, the transition between the states of robot, resumption from deadlock and so on. This mechanism has resolved the dynamic task allocation problems of a large scale of robots, and satisfied some constraints too. The innovations of this chapter are introducing the transition between missions into the market mechanism to optimize the allocation results and designing a coordination policy based on impatience to resolve the deadlock problem which happens easily when the scale of robots is small.(2) The traditional obstacle avoidance algorithms are most based on the manually designed rules, but these algorithms are lack of intelligence and unable to work in the unpredictable environment beforehand. It can improve the adaptability of robot to the environment if learning mechanism is adopted, and it is one of the abilities of robot in multi-robot system. So a layered reinforcement learning method is proposed to learn obstacle avoidance rules, and this method decompose the obstacle avoidance behaviors into three simple child modules– static obstacle avoidance module, dynamic obstacle avoidance module and behaviors fusion module, then these modules are trained separately. It reduces the state space of the learning system, accelerates the learning procedure. And it avoids the imprecision of the obstacle avoidance policy that arises from the fusion of the simple behaviors by taking the outputs of simple behavior learning modules as the inputs of the fusion learning module. The simulation results prove the convergence and validity of this method, and it makes the robots can coordinate independently. This method also improves the intelligence of the obstacle avoidance policy, and accelerates the learning procedure relative to traditional learning methods.(3) The effect of communication on swarm robotic system is huge. The robots spread all over the environment and their knowledge is not the same, so they can exchange useful information by communication to improve efficiency. So it's worth researching on how to accelerate the learning using the communication between robots. This paper proposes a cooperative learning method based on policy maturity, which uses policy as the estimation and communication unit, and designs policy maturity estimation method for it. This method takes fully advantage of the learning results of other robots, and improves the learning rate. The simulation results verify this method.(4) Introduces some self-organized pheromones and models of some biological colonies, and their application in swarm robotic system. Then the forage mission is modeled using Eulerian modeling method, and is analyzed subsequently. The analysis result is that the time taken to complete the mission is not always shorter while the number of foraging robots in environment became larger, and the optimal number of foraging robots correlates with the environment. This paper analyzes the impact of the parameters of robot on the time taken to complete the mission. The results are validated by simulation results.Summarily, this paper accomplishes some foundational theory research work for the application of swarm robotic system, which includes the implement method of coadaptation and the modeling and analysis of swarm robotic system. However there are some other problems to be researched and resolved, such as the proof of the convergence of cooperative learning method based on policy maturity, and the mathematical analysis method for swarm robotic system needs to be consummated.