Research On Navigation Of Indoor Mobile Robot Based On Deep Reinforcement Learning

With the continuous development of artificial intelligence technology,indoor mobile robots are undertaking increasingly complex tasks in industrial and family environments.In practical application,indoor mobile robot needs to avoid obstacles in real time and realize autonomous navigation in unknown,complex and dynamic environment.In the process of navigation,users usually put forward higher requirements for navigation efficiency and energy consumption in addition to requiring robots to avoid obstacles safely and reach designated targets.Inspired by this,this paper designs a Deep Reinforcement Learning algorithm based on reward decomposition and memory enhancement.The method learns the multi-dimensional control strategy by decomposing and designing the reward function,and introduces the memory module to enhance the memory ability of the robot,so as to realize the obstacle avoidance navigation of the indoor mobile robot in the unknown,complex and dynamic environment.It can directly learn the features significantly associated with the action value function from the visual input and realize the direct mapping from the first visual image observation to the multi-dimensional navigation strategy.Experimental results show that this method can reduce energy consumption on the premise of avoiding obstacles and reaching the target quickly,and the robot can also make appropriate decisions according to the prior sequence information in the dynamic environment.The main work of this paper can be summarized as follows:Firstly,a novel algorithm for robot autonomous obstacle avoidance based on Deep Reinforcement Learning was proposed for obstacle avoidance tasks.In particular,for the domain where the optimal value function is difficult to be effectively reduced to a low-dimensional representation,this paper proposes a new reward decomposition architecture.The multi-dimensional obstacle avoidance strategy,which is difficult to learn,is divided into several sub-tasks,namely,speed control branch and direction control branch,and each branch contains obstacle avoidance reward knowledge.Each subtask branch can learn its own value function by decomposing and designing its own reward function.This structure makes full use of deep learning feature mapping,so that each sub-task can better learn the empirical knowledge of a specific domain,make more appropriate obstacle avoidance decisions,and reduce the learning difficulty of obstacle avoidance strategies.The experimental results show that this algorithm can perform autonomic obstacle avoidance in unknown static and dynamic environments,and has better performance in three performance indexes: speed,number of direction change and number of exploration steps before collision.Then,a Deep Reinforcement Learning algorithm based on memory enhancement is proposed for mapless navigation tasks.Based on the concept of partial observability,a memory module is designed to enhance the memory ability of the robot by introducing the Long Short Term Memory.It collects information over time and selects a reasonable updating method to assist the robot to track historical information for a long time.Therefore,the robot can use the useful information in the environment to make better navigation decisions and avoid the dynamic obstacles in a predictive way.Considering the lack of depth information in monocular color images,the full convolutional residual network is introduced to predict the depth of the image,and the predicted depth map is used as the input of the Q network to improve the performance of the algorithm.Through the framework of reward decomposition,the corresponding reward functions of three branches,namely speed control,direction control and target approach,were designed to learn the multi-dimensional navigation strategy,and the method of branch fusion was proposed.Experimental results show that the proposed algorithm can reach the specified target in unknown,static and dynamic environments,and its navigation efficiency and energy consumption are better.Finally,three-dimensional indoor simulation environments of robot obstacle avoidance and navigation is built under the framework of robot operating system using Gazebo.A mechanism for subscribing and publishing topics is created for the communication process.Taking Turtle Bot as simulation object,the feasibility of the proposed algorithm is verified by visualization method.