Study Of Robot Navigation Based On Deep Reinforcement Learning

With the popularity of sweeping robots,service robots and AGVs,the autonomous navigation of indoor mobile robot has become a research hotspot again.In order to perform increasing complex tasks,indoor mobile robots must adaptively implement obstacle avoidance and navigation according to perceived environmental information.In recent years,Deep Reinforcement Learning(DRL)has achieved great success in artificial intelligence games and robot control,and is naturally introduced into robot navigation.In this paper,obstacle avoidance and navigation of the mobile robot,which is equipped with laser radar to sense the environment,is studied based on deep reinforcement learning.First,an indoor simulation system of obstacle avoidance and navigation is built based on ROS and Gazebo.The system consists of two indoor simulation scenes and a TurtleBot mobile robot.The two indoor simulation scenes are: narrow and tortuous "L" type scene for static obstacle avoidance;scene for dynamic obstacle avoidance and driving to random target.A lidar sensor is equipped for the TurtleBot robot to capture the robot's own distance from the surrounding environment.The state information about robot and lidar data are sent to module of deep reinforement learning.Next,based on the built-up "L" type simulation scene,the static obstacle avoidance with discrete action output and continuous action output is studied separately.The state input of the DRL algorithm is Lidar distance data,and the action output is an angular velocity value that controls the movement of the robot.It is difficult to adjust parameter,and ensure stability of algorithms in DRL.For the discrete action output situation,the static obstacle avoidance based on DQN,DDQN and Dueling-DQN algorithms is implemented respectively.The experimental results show that the three algorithms can perform static obstacle avoidance,and the advantages and disadvantages of the three algorithms in static obstacle avoidance are gave by comparision.For continuous action output situation,static obstacle avoidance based on DDPG algorithm is also completed.Simulation results demonstrate the feasibility of DDPG algorithm for static obstacle avoidance.At the same time,the comparison between DDPG and that three DQN algorithms,shows that the DDPG algorithm can achieve better angular velocity control and smoother trajectory for the robot.Then,we study progressive and easy-to-difficult training strategies to combine DQN with transfer learning in navigation.In the first step,based on DQN,the robot has the ability to reach any target point in simulation scene with only navigation target point and no obstacle.In the second step,two static obstacles are placed in the first simulation scene.The neural network structure and weights in the DQN model,from the first step training,are transferred to the new simulation scene.Only after a small amount of training on the robot in the new scene,the robot learns to avoid static obstacles and reach random target.In the third step,the static obstacles in the simulation scene start random walking.And the transferring mechanism similar to the second step is also adopted to enable the robot to have the ability of avoiding the dynamic obstacle and reaching random target.Finally,the TurtleBot2 robot platform is built,and the traditional Slam method and deep reinforcement learning method are used to perform the indoor obstacle avoidance navigation.It shows that deep reinforcement learning manages to be applied for navigation of real robot.