Research On Path Planning Algorithm Of Unmanned Vehicle In Coal Mine Based On Reinforcement Learning

Underground unmanned vehicles are an important part of smart mines,and pain planning is an extremely important part of unmanned tasks.Due to the characteristics of narrow tunnels,many obstacles,scattered and easily changeable working sites in coal mines,traditional path planning algorithms will have problems such as low planning efficiency,real-time performance and poor planning quality.Reinforcement learning can allow the agent to learn the surrounding environment through "trial and error",so as to maximize the benefits.Therefore,this paper proposes the application of reinforcement learning to the improvement of the path planning algorithm,which improves the real-time performance and the path planning of unmanned vehicles.The adaptability also ensures the quality of the path.The main work includes the following aspects:(1)In the aspect of global path planning algorithm,a Q-RRT algorithm combined with reinforcement learning algorithm is proposed.Aiming at the problem of low node sampling efficiency in the RRT(Rapidly-exploring Random Trees)algorithm,the method of designing a reward function is used to guide node expansion,which improves the algorithm search efficiency.At the same time,the pruning method and the cubic Bezier curve with constraints are used to optimize the generated path,the result shows that the algorithm improves the efficiency of path planning and realizes the smooth processing of the path.(2)In terms of local path planning algorithm,a GAIL-D3PG(Generative Adversarial Imitation learning-D3PG)algorithm was proposed based on deterministic strategy and Generative Adversarial Imitation Learning.First,the Double Experience Replay DDPG(D3PG)algorithm was designed to increase the expert experience replay pool,and the experience replay pool used in DDPG(Deep Deterministic Policy Gradient)algorithm was improved and the efficiency of exploring the environment is accelerated.Then,on the basis of D3PG algorithm,GAIL-D3PG algorithm is designed by combining generative adversarial imitation learning,which uses expert data to directly learn expert strategy,further improving the learning efficiency of the algorithm.The experiments show that the learning efficiency and training effect of the GAIL-D3PG algorithm are greatly improved compared with other algorithms.(3)The algorithm in this paper is verified by roadway simulation under the ROS(Robotic Operating System)platform.The roadway simulation environment was built by using the Gazebo simulator,and the Turtlebot3 mobile robot was selected as the simulation robot for training,and the algorithm in this paper was transplanted to the Turtlebot3 entity robot,and the feasibility of the method in this paper was verified in the simulated roadway and the real environment.The Q-RRT algorithm and GAIL-D3PG algorithm proposed in this paper not only improve the quality of underground unmanned vehicle path planning,but also can be widely applied to underground rescue,inspection and path planning in complex ground environment,which has certain theoretical and practical significance.