Research On Improvement Of Motion Coordination Algorithm Based On Reinforcement Learning In Specific Road Network Environment

In recent years,with the outbreak of the fifth information technology revolution,robotics,artificial intelligence and other technologies have developed rapidly.Intelligent robots also play an increasingly important role in daily life,and are widely used in logistics,express delivery,catering,supermarkets and other scenarios.Due to the multiple and complex scene tasks,it is often necessary for multiple robots to cooperate with each other to complete the task.The coordination between multiple robots is one of the core problems that cannot be avoided.In this paper,the vehicle-based mobile robot is taken as the research object.In a specific two-dimensional road network environment,an agent is trained by using the deep reinforcement learning algorithm.It can control multiple robots in the process of motion without collision,and can complete their own tasks in a short time.In this paper,a Multi-Loss Double DQN(MLDDQN)algorithm is first proposed,which is based on the Deep Q-Network(DQN)algorithm and a target network pool is used to store the trained target network of group K.The weight of the target network is inversely proportional to the storage time of the target network.In a new round of iterative training,it weights the loss of the K-group target network to get a new loss,and updates the target network pool.Compared with the DQN algorithm,the MLDDQN algorithm can reduce the estimated variance of the target network value function,make the neural network training more stable,and accelerate the training speed.Then,this paper combines the MLDDQN and multi-robot motion coordination problems to construct the state space of the robot road network environment,formulate a discrete action space and design a reward function model.Through repeated iterative training to obtain an agent,a set of feasible motion coordination strategies can be generated.Although the MLDDQN algorithm can solve the problem of multi-robot motion coordination,it discretizes the robot's movements.In the actual scene,the robot's motion is a set of continuous actions.Therefore,this paper also proposes a multi-robot motion coordination algorithm combined with Deep Deterministic Policy Gradient(DDPG)algorithm.The algorithm uses the continuous speed value of the robot to design the action space,and re-enacts the state space and reward function model.After iterative training,a set of feasible motion coordination strategies can also be obtained.Compared with the discrete motion method,the motion coordination strategy obtained by the algorithm is more in line with the actual situation,and the robot's motion sequence is better.In order to verify the correctness and efficiency of the proposed algorithm,the MLDDQN algorithm,DDPG algorithm and mainstream DQN,DDQN,and Averaged DQN algorithms are compared and analyzed on coordinated tasks involving different paths.The experimental results show that the motion coordination strategy trained by the MLDDQN algorithm can effectively avoid the collision of multiple robots during the movement process,and the convergence speed is significantly better than other algorithms.The experimental results of the DDPG algorithm are better than the discrete action algorithm,which effectively makes up for the shortcomings of the discrete action,and has practical application significance.