| Recently, multi-robot system is researched by more and more scholars and applied in broader domains because of its advantages, such as efficiency, robust and distribution. Maintaining formation is used to achieve the mission and enhance the performance of the system in many domains. The main idea of multi-robot formation control comes from the nature. Formation behaviors in nature, like flocking and schooling, benefits the animals that use them in various ways. By maintaining a certain formation, animals combine their sensors to maximize the chance of detecting predators or to more efficiently forage for food. Maintaining formation can enhance the performance of the system and accomplish complicated task which cannot be done by a single robot. Maintaining formation is very important, especially in the military applications. Therefore, multi-robot system formation control is a research with theoretical and practical significance.In recent years, reinforcement learning has become one of the key research areas in artificial intelligence and machine learning. It has attracted many researchers in other fields including: operations research, control theory and robotics. Reinforcement learning has the advantages of little priori knowledge, so it is suitable to the learning of control programming. With the development of the method and theory research, reinforcement learning is becoming more and more useful in the domain of engineering optimization and control problem, especially the robot system.The work in this paper is supported by National Natural Science Funds Program"co-adaptation theory and technology on swarm robotic system under complex environment"(60675057). The main research area is the formation control strategy using reinforcement learning and behavior-based method. The main research about this dissertation is as follows: (1)Several formation control methods and their advantages and drawbacks are presented in this paper. Behavior-based formation control method is chosen to be the main research direction. Five primitive behaviors called move_to_goal, avoid_obstacle, swirl_obstacles_goal, maintain_formation and noise are introduced for the formation control strategy. Neighbor-referenced techniques for formation position determination have been adopted. Using Teambots simulation system, several formations like line, column, diamond, wedge and phalanx are simulated. By designing the control strategy, the formation figure can be changed according to the environment and avoiding stable and dynamic obstacles at the same time. Aiming at the line-formation, single-neighbor-referenced and double-neighbor-referenced approaches are used, and double-neighbor-referenced approach performs better. Simulation experiments indicate the feasibility and validity of the control strategy.(2)In order to improve the adaptability of behavior-based method, this paper proposed a formation control method using reinforcement learning, which enable the robot formation to find a best formation strategy in the unknown environment. The formation control system is divided into two levels. Reinforcement learning is used in the upper level to learn the formation strategy. The lower level is using the behavior-based method to perform the formation according to the learned strategy. The method has the function to lower down the dimension of state space and shorten learning period. The environment in the paper is a narrow channel with different width. After many times of learning, robots can learn how to keep best formation. When the width of the channel is changed, robots can change formation autonomously to a suitable formation with biggest channel cover rate. The simulation results verify this method.In a word, this paper accomplishes some foundational theory research work for the multi-robot formation control strategy. However, there are still lots of work to be researched and resolved, such as the problem of distributed reinforcement learning and reinforcement learning in the unknown environment. |
PDF Full Text Request