Research On Positioning And Navigation Of Mobile Robot Based On Reinforcement Learning

With the rapid development of unmanned driving technology,mapless navigation of robot with unsupervised learning,real-time planning,and autonomous decision-making has been brought into sharper focus,while the map-based navigation of robot can not meet the technical requirements that navigation without prior map information.As a result,this work focuses on map-free navigation of mobile robot using reinforcement learning.The primary work is as following:First of all,3D simulation environment for mobile robots used to train reinforcement learning models was designed and built in order to significantly improve training efficiency and reduce the economic losses caused by robot collisions in real model.We chose the Gazebo simulation platform with a powerful physics engine by comparing three different simulation environments.We then built a mobile robot model and eight representative and realistic simulation scenarios,and the functions of Markov decision-making and robot scene interaction was provided by using Gazebo’s service communication.Secondly,the obstacle avoidance strategies for mobile robots using reinforcement learning was studied in the simulation environment.By analyzing the requirements of obstacle avoidance used in mapless navigation,the state space,action space,and reward function were designed,and two obstacle avoidance strategies based on the PPO algorithm and the deep Q network respectively are proposed.Different avoidance training scenarios including dynamic obstacle scenario and rectangular-ambulatory-plane obstacle scenario were designed and built to compare the performance of the proposed methods.The experimental results demonstrated that the PPO algorithm achieved a success rate over 80% in obstacle avoidance studies,.The success rate is 14 percents higher and the training period is 40 percents shorter than that of the DQN algorithm.Moreover,end-to-end navigation methods for mobile robots based on reinforcement learning was studied after the research of obstacle avoidance strategies.By comparing the decision strategies used in obstacle avoidance and navigation of mobile robots,end-to-end navigation strategies based on Proximal Policy Optimization and a deep Q network are proposed respectively.Multi-information fusion based reinforcement learning network and composite reward function are presented to address the problem of low navigation efficiency.A continuous reward function of M-PPO is developed to improve the training efficiency of the PPO algorithm.The experiments on six scenarios were conducted to verify the effectiveness of the proposed algorithms.The experimental results showed that the PPO algorithm based navigation has a success rate up to 91 percents,which is 30 percent higher than that of DQN,and the training time is halved;the training efficiency of the algorithm employing the continuous reward function of M-PPO was improved by two to three times and all of the SPL navigation indicators rose over 0.67.Finally,an experimental research of simulation strategy migration was conducted in this work.The problem induced by that the dimension of the actual robot lidar and the dimension of the strategy model is different was overcome by using the laser cutting principle.The odometer information is presented to locating a real robot in a mapless scene.The Vicon system was used to capture the motions of robots and pedestrians in the obstacle avoidance test,which solved the problem of path visualization in a mapless setting.Due to the employment of lidar which provided the observation value in the obstacle avoidance experiment,the response speed of obstacle avoidance reached 25 ms.In ten navigation trials,single-target navigation reaches the target point 8 times,while multi-target navigation reaches the target point 7 times.Statistical results showed that mobile robots can well achieve mapless navigation in specific scenarios.