Research On Flocking Control Problem Of Mobile Robots Based On Reinforcement Learning Method

With the maturity of robot technology,people have higher requirements for the complexity of tasks completed by robots.In some special environments,the processing ability of a single robot is no longer enough to satisfy us,so more attention is paid to multi-robot system.Swarm robots have more advantages than single robots in completing complex tasks,and their more stable system can also ensure that tasks have a higher degree of completion.In an unknown environment,there is many uncertain factors,and the robot lacks a priori knowledge of the environment,so it is required that the swarm robot should have strong adaptability and reaction speed.Flocking control method based on reinforcement learning can enable robots to explore information independently in the environment.The environment will evaluate each action took by robots and the robots will modify their action generation network.In such a trial-and-error process,the robots will learn valuable experience.In unknown complex environment,swarm robots with strong self-learning ability play a very important role in realizing swarm movement.The research background and practical significance of flocking control for mobile robots based on reinforcement learning are firstly described in this dissertation.Then,the current research status of swarm control and reinforcement learning at home and abroad is analyzed.The tasks and structure of the dissertation that need to be completed are introduced in the end.In this dissertation,the flocking control problems of multiple mobile robots in discrete motion space and continuous motion space are studied based on reinforcement learning method,and the simulation experiments are implemented in an environment with obstacles and a barrier-free environment.The main work is divided into the following four parts:Firstly,the action function of the robot is fitted with the neural network,and the output of the neural network is used as the guidance of the robot action.It is expected to realize the clustering state by achieving the consistency of the direction and speed of the multi-robot movement.On account of lacking sufficient training samples,features learned by the neural network are too few.Then the cluster motion state is not stable,and it is going to reach a situation where robots are moving to different directions.Compared with supervised learning method,which often requires millions of data,reinforcement learning only needs thousands of data to complete the training of the model.Secondly,in the discrete motion space,the flocking control of mobile robots is realized by DQN algorithm.The simulation environment of two robots is built on the Gym platform.Kinematics model of the robots is established.Besides,the reward mechanism is redesigned.Appropriate training plots and super parameters are set either.The robots could only produce discrete movements.At the end of the training,according to the loss curve of the network and the potential energy curve,the clustering state of the robots is evaluated.The experimental results show that the two robots can maintain a stable relative position in the end,which verifies the effectiveness of the algorithm for flocking control.Thirdly,flocking control of multi-robot is realized by modified MADDPG algorithm in the continuous motion space.Multi-Agent Particle Environments developed by Open-AI team is used as a test environment.Action network is built with two whole connection layers.According to the state of aggregation between robots,the distance between robot and target,and collision frequency,we redesign the reward function.Based on the change curve of reward value and potential energy function value obtained by robots in the environment,the clustering state and network convergence rate of robots are evaluated.The simulation results show that the robots can keep the aggregation state without collision in the group when moving to the target point,and also verify the effectiveness of the algorithm to realize the swarm movement in the continuous motion space.Finally,based on the MADDPG algorithm,the multi-robot clustering control is realized in the environment with obstacles.The robots can be guided to move toward the target point in a swarm state while avoiding obstacles and neighboring robots according to reward mechanism which has been set.After the training,the saved training data about the network will be loaded to shown the animation,and the reward curve and potential energy curve of the robot in the process of movement will be drawn.The work completed in this dissertation verifies the effectiveness of the reinforcement learning-based algorithm for swarm motion control in obstacles.