Mobile Robot Navigation Based On Deep Reinforcement Learning In Dynamic Dense Crowd Environments

In the past decades,with the rapid development of the robot technology,mobile robots have gradually entered into human daily life,providing services in a variety of pedestrian intensive scenes,such as supermarket shopping guide,restaurant delivery,etc.The dynamic dense crowds pose many challenges for the navigation of mobile robots.How to make robots efficiently plan collision free paths,abide by the walking norms of human and respect the human intension,is the research hotspot in the field of mobile robots,which has an important practical significance and research value.Most of the traditional navigation algorithms are based on reactive obstacle avoidance algorithms,which treat pedestrians as simple dynamic obstacles,without considering their intention and social norms.It can not meet the safety and social requirements of mobile robots in crowd navigation.To solve this problem,this dissertation explores and studies the pedestrian intention cognition,human trajectory prediction and reinforcement learning navigation algorithm,and proposes a reinforcement learning navigation algorithm based on spatial graph attention mechanism.The main contributions are as follows.Firstly,a spatial-temporal representation of the crowds for the topological relationship between robots and pedestrians is established by gridding.Then,according to the attention mechanism,a spatial graph attention mechanism which can adaptively aggregate among weights and extract pedestrian spatial information is designed to model the interaction between pedestrians.Then,the temporal convolution network is used to gather the current and historical state information to enhance the cognitive ability of the robot to the pedestrian behavior intention from the space-time dimension,so that it can give different attention weights to the pedestrians in different states,and provide the feature of crowds,including the interaction among the pedestrians for the subsequent reinforcement learning decision-making module.Secondly,in order to improve the reactive navigation algorithm which is prone to produce inefficient path,this dissertation studies the human trajectory prediction.The state information of pedestrians is represented in the form of spatial-temporal graph,and the spatial features including pedestrian's interaction are extracted by the attention mechanism of spatial graph.In the temporal dimension,to improve the realtime performance of prediction,a lightweight one-dimensional time convolution with gating is designed to replace the traditional long short-term memory network.Through the comparative analysis of experiments on public data sets,it is confirmed that the pedestrian trajectory prediction network has a good performance in accuracy and time-consuming.Then,on the basis of pedestrians' intention cognition module and trajectory prediction module,a crowd navigation algorithm based on deep reinforcement learning is designed,and the navigation problem in crowd is abstractly modeled as a Markov decision-making process.Secondly,a multi-layer fully connected network is designed to fit the state value function of the reinforcement learning.According to the predictive trajectory,a decision-making module combined with multi-step prediction is proposed,which makes the value network evaluate the current state value more accurately,so as to select better actions and improve the efficiency and success rate of navigation.At the same time,social norms are included in the design of reward function,so that the robot's decision-making is more resonable.Finally,the simulation of the algorithm is carried out and the results are compared and analyzed.Firstly,a simulation environment is built for algorithm training and testing.Secondly,in the simulation environment and public crowd data set,the navigation scheme based on deep reinforcement learning is compared with the current representative navigation algorithm in the crowd.The simulation results show that the proposed method performs better in the navigation success rate,path time-consuming and sociality,and can make the robot in the crowd.The effectiveness of the method is verified by efficient and social navigation.