Research On Application Of Reinforcement Learning In Autonomous Navigation Of Mobile Robots

In recent years,robotics has increasingly applied in various fields,and has become the main focus in academia and industry.Robot technology is a comprehensive combination of modern scientific theory and practice,so it combines multiple disciplines and technologies.Among all types of robots,autonomous mobile robot is an important branch of robotics.Autonomous mobile robot is widely involved in industrial,agricultural production and social services.Mobile robot is a combination of various technologies,such as sensor technology,driving technology,infrared technology and mechanical technology and so on.But in these related theories,navigation is the core of its research.It was created for robotics and is a hot spot in academia and industry.Robot navigation is the technical foundation of all autonomous navigation robots,and it is also a top priority.All robots that need to complete a certain task have to complete the navigation task firstly.For example: an indoor sweeping robot needs to complete the indoor construction firstly,then plan the path and navigate,finally complete the task of sweeping floor;Another example is security robot,which complete various functions base on navigation,such as pedestrian detection and walking along specified routes.This thesis mainly proposes two robot navigation algorithms with the help of reinforcement learning and deep learning technology,finally verifies them in a robot 3D simulation environment.The main work of this thesis is described as follows:(1)The construction process of the simulation environment and the optimization of the simulation environment are explained,finally I tested its correctness.(2)Starting from the most classic reinforcement learning algorithm Deep Q-learning,I firstly introduced the success of the original model and learned from it.I have maintained the superiority of DQN in image processing,and proposed an improvement plan to improve the DQN or further convolution structure in processing sensor data such as radar and compass.Finally,most of the reinforcement learning environments are based on the third perspective,there is no possibility of knowing the global information in the actual robot navigation,so I only have local information in the simulation environment which I made with ROS and Gazebo.Since the information obtained for the robot is in the case of local information,I add a LSTM network.Finally,I verified above methods in a simulation environment.(3)The third part focuses on the robot's navigation in continuous motion space.Considering the continuity of robot action execution,I first analyzed the Deep Deterministic Policy Gradient to find out the limitations of the DDPG algorithm on robot navigation,and learned from the radar data processing method in the previous chapter.Through further analysis,I designed an action space different from discrete space,and take the time step as one of the network outputs to further refine the action execution time.In addition,I designed a new reward function to adapt to continuous action space.It is finally verified in a simulation environment.Through the above research and implementation,and experiments in a simulation environment,I verified the feasibility of applying reinforcement learning to robot navigation.