Research On Efficient Autonomous Exploration Strategy For Mobile Robots Based On Deep Q Network

In recent years,with the improvement of science and technology,people pay more attention to robot intelligence,including autonomous exploration and map construction of mobile robots in unknown environments.In an autonomous exploration task in an unknown environment,the robot needs to find the optimal path with constraints.These constraints include: the path cost,the map construction accuracy,and so on.The existing autonomous exploration methods for mobile robots are based on frontier,information theory and machine learning.The methods based on frontier and information theory adopt greedy strategies and cannot achieve global optimization result.Intelligent methods based on machine learning mostly construct exploration candidate targets by sampling,which not only makes it difficult to guarantee the training efficiency of the algorithm,but also causes the robot to fall into a dead end during the training process.At the same time,none of these methods provide a good solution to deal with map occlusion in the process of robot autonomous exploration.In response to the above problems,we design an exploration method based on deep reinforcement learning and local map prediction,so that the robot can predict the structure of the environment and plan the order of exploration targets efficiently.We first design the robot's efficient autonomous exploration system architecture,including localization and mapping thread,local map prediction thread,exploration target planning thread and path planning thread.Finally,a single-line Li DAR is used as a sensor to construct a grid occupied map.In the part of local map prediction based on deep generation model,we design a local map prediction neural network based on the variational autoencoder,which converts the prediction of map obstacles into a binary classification problem of pixels using binary cross entropy.The network is trained with the Kullback–Leibler divergence loss function,and evaluated with the prediction accuracy and intersection-over-union during the exploration process.In the part of exploration target planning based on deep reinforcement learning,we combine deep Q-network with Long Short-Term Memory network structure to design the exploration target planning network.The action space is based one the geometric center of frontier and the state space is based on the predicted local map.A dense reward function is designed according to the length of the path and the information gain obtained,which solves the problem of low efficiency of reinforcement learning training and falling into dead end.The new system provides better performance for robot autonomous exploration in the time-series decision-making scenario of exploring target planning.Finally,the comparison and analysis with three representative traditional exploratio n methods in a simulation environment verify that the autonomous exploration method based on deep reinforcement learning and map prediction has better performance.At the same time,this method also has generalization ability.