Research Of Path Planning Algorithms Based On Deep Reinforcement Learning

This paper aims to replace the mechanical path planning algorithm with intelligent algorithm which integrates perception and decision.Deep learning is represented by neural networks stacked at multiple levels.Neural networks are fully differentiable,distinguish from algorithms based on mechanical instructions.Due to the lack of understanding of the problem itself,a mechanical algorithm is unable to generalize to different tasks.Many traditional algorithms are replaced by the intelligent algorithm based on end to end training models.In general,solving a robot's automatic path finding problem needs the image of road condition,recognition and understanding of spatial information,annotating it into a map,and finally path planning.At present,image recognition and computer vision are very dependent on the intelligent algorithms based on deep learning,which can make computer understand image information,while the path planning relies on the mechanical algorithm,such as the shortest path planning algorithm,requires additional image processing and data access.The gap between image perception and path planning is huge,resulting in inefficiency.This paper combines reinforcement learning with deep learning,analyzes their functionality of applying on path planning problems,proposes a modified algorithm based on deep reinforcement learning for path planning.The main idea is constructing a map from input image to output path by deep neural network.Details of the article are stated as follows.First,this paper studies deep Q network(DQN),using a single neural network for both image handling and action-value calculation,combing deep learning and reinforcement learning together.Specially,the neural network is a fully convolutional neural network with visual attention.Second,due to the short slab on dealing with multi-step decision making and planning,a value iteration module inspired by value iteration network(VIN)is added on the network.The value iteration module is modified to ease cumulative error.Finally,the action-value function is replaced by advantage function and state-value function which leads to a dueling architecture.This paper uses 2D grid environment for experiment and imitation learning for acceleration,using expert's planning results instead of agent's experience during training,and use shortest path finding problem to measure algorithm's performance.The experimental results show that the proposed path planning algorithm is obviously better than the same type algorithm based on DQN and VIN.The proposed algorithm has the same operating efficiency as VIN.This paper not only improves the existing model but also provides an architecture for applying deep reinforcement learning to path planning.