Research On Path Planning Method Of Mobile Robot Based On Deep Reinforcement Learning

Path planning is one of the main research contents in the field of mobile robot.Path planning of mobile robot means that the robot chooses a collusion-free path from the starting point to the ending point in its environment.Traditional path planning methods mainly rely on environment maps.With the rapid development of Deep Learning(DL)and Reinforcement Learning(RL),the Deep Reinforcement Learning(DRL)algorithm is widely used in path planning and obstacle avoidance of mobile robots.In this thesis,a two-wheeled differential mobile robot is studied,and deep reinforcement learning algorithm and improved method are used to realize the path planning of mobile robot.The main research contents of this thesis are as follows:(1)In the Gazebo emulator environment of Robot Operating System(ROS),path planning of mobile robots based on Deep Q Network(DQN)method is realized.The kinematics analysis of the two-wheeled differential mobile robot is carried out,and the simulation environment is built under ROS.The environment and obstacle information are perceived by Lidar,the reward function is improved,the state space and action space are designed,and the end-to-end path planning of the mobile robot is completed based on DQN method.The ability of autonomous path planning based on DQN method is verified by simulation experiments.(2)On the basis of robot path planning based on DQN method,DDQN(Double DQN)method is used to solve the shortcoming of overestimating Q value by DQN method.Long Short-Term Memory(LSTM)network is added to solve the problem of poor robot decision-making effect caused by partial observation.A continuous reward function with Heuristic Knowledge(HK)is designed to make the reward complete and continuous,and static and dynamic simulation environment is built under Gazebo environment to realize the path planning of mobile robot based on DDQN-LSTM-HK.The simulation results show that the path length obtained by LSTM-DDQN-HK method is reduced by 8.58%and 6.98%respectively compared with DDQN method in static and dynamic environments.The memory network and the heuristic knowledge auxiliary reward improve the decision-making efficiency of the mobile robot,so as to plan a better path.(3)In order to improve the planning efficiency in large space,global path planning of ant colony algorithm is combined with local path planning of deep reinforcement learning.Global path planning in large space is realized based on ant colony algorithm,and the ant colony path information is added into the improved reinforcement learning reward function to realize robot path planning.And a path tracking controller based on Deep Deterministic Policy Gradient(DDPG)method is designed.Static environment simulation experiments show that the ant colony-reinforcement learning method can effectively realize path planning in large space area.The simulation experiment of new obstacle environment shows that the ant colony-reinforcement learning method can successfully avoid new static obstacles and complete the path planning and tracking task of mobile robot.Compared with the reinforcement learning method,the path length of the ant colony-reinforcement learning method is reduced by 13.77%and 16.02%respectively in the static environment and the newly added obstacle environment,indicating that the improved method can make up for the deficiency of reinforcement learning and plan a better path in the large space area.