Research On The Local Path Planning Of Mobile Robot Based On The Deep Reinforcement Learning Under ROS

Autonomous navigation technology is one of the core and focal points of mobile robot applications in unknown dynamic environments.Robots complete the tasks such as mapping,positioning,sensing,and path planning through their own installed sensors,among which path planning is a key technology.Traditional path planning algorithms lack the ability of autonomous learning in the face of dynamic unknown environments,making it difficult to accurately and quickly plan a collision-free path from the starting point to the target point in different environments;and the limited environmental information obtained through the sensors leads to the problems such as poor real-time performance,path redundancy,and local deadlock in planning paths.Deep reinforcement learning algorithms are widely used in local path planning for mobile robots in unknown dynamic environments because they do not rely on environmental modeling and own autonomous learning capabilities.This paper proposes the improved algorithms to solve the problems of deep reinforcement learning algorithm in the application of local path planning for mobile robots,which enables robots to complete planning tasks independently and efficiently.Aiming at the problems of slow convergence speed and unstable model of the Deep Deterministic Policy Gradient(DDPG)algorithm in the local path planning of mobile robots,this paper introduces an APF-LSTM-DDPG algorithm based on DDPG algorithm with the Artificial Potential Field(APF)algorithm and introducing the Long Short-Term Memory(LSTM)structure.Firstly,LSTM structure is added to the DDPG neural network structure,and higher samples will be rewarded with priority learning through memory unit and forgetting unit to improve the sample utilization.Secondly,the potential field reward and punishment function are designed to speed up the learning of obstacle avoidance strategies.Then,the resultant force of the potential field is used to adjust the angular velocity to guide the robot to the target point.Transfer learning technology is used to train the robot from a simple environment to a complex one gradually to speed up the training.Finally,the algorithm is simulated and verified in different simulation environments based on the simulation platform Robot Operating System(ROS).The simulations show that the reward value of APF-LSTM-DDPG algorithm can be more stable during training.It can improve the success rate of the algorithm and reduce redundant sections in the planned path.Aiming at the problems of low sample utilization,large exploration space and path redundancy of Soft Actor-Critic(SAC)algorithm in local path planning of mobile robots,PER-SAC algorithm and PER-SAC-APF algorithm are proposed.Firstly,this paper proposes the PER-SAC algorithm based on the technology of Priority Experience Replay(PER).The algorithm extracts the samples from the experience pool by the priority instead of equal probability,so that the network prioritizes the samples with larger errors,and improves the convergence speed and stability of the robot training process.The algorithm optimizes the calculation of Temporal-Difference(TD)error and reduces the training deviation.At the same time,transfer learning is used to improve the efficiency of model training.In addition,the algorithm designs an improved reward function to increase the robots’ intrinsic reward.After that,in view of the low smoothness path of PER-SAC algorithm,the PER-SAC-APF algorithm is proposed by introducing the artificial potential field method,which modifies the action selection of the algorithm model in path planning.Finally,the simulations on the ROS platform show that the PER-SAC algorithm can reduce training time and outstrip the original algorithm in convergence and path planning performance.PER-SAC-APF algorithm can effectively reduce redundant paths and improve path smoothness.The designed APF-LSTM-DDPG and PER-SAC-APF algorithms are downloaded to the physical robot,and are tested and verified the feasibility and effectiveness of them in real operating environments.