Reinforcement Learning Robot Navigation Algorithm For Diverse Dynamic Environments

As the use of robots continues to penetrate into people’s lives,the requirements for robots to navigate in complex dynamic environments are also increasing.This makes traditional robot navigation algorithms unable to meet the needs of these scenarios.Due to the amazing results achieved by reinforcement learning on sequential decision-making problems in recent years,some scholars have begun to use reinforcement learning to solve navigation problems.But in different navigation scenarios,there are still many scientific questions about the state space,reward function,and training process of reinforcement learning navigation tasks.In this thesis,multi-robot scenarios and social pedestrian scenarios are selected for research based on the application of mobile robots in the real world.The multi-robot navigation problem requires multiple robots with the same strategy to avoid other robots and obstacles in the environment when completing the navigation task.This paper proposes a multi-robot navigation algorithm based on grid map,which takes the sensor grid map as the state space of the robot.Sensor map can be generated by depth sensors with different parameters and can represent obstacles and other robot information,allowing multi-robot navigation algorithms to handle both static and dynamic obstacles.We use multiple robots to perform navigation tasks in different random environments to collect data for training,which improves the generalization of the navigation algorithm.Experiments show that the multi-robot navigation algorithm based on grid map proposed in this paper has better navigation performance than other reinforcement learning multi-robot navigation algorithms and traditional robot navigation algorithms,and has certain robustness to sensor noise.In the social pedestrian environment,the robot needs to reasonably deal with pedestrians moving with various strategies in the scene.This introduces two problems for the navigation algorithm,one is the representation of pedestrian information,and the other is the response to pedestrian movement strategies.Aiming at the problem that the raw sensor information alone is insufficient for pedestrian perception,We fuses pedestrian information and sensor information.We proposes a pedestrian map,which maps the position and speed information of pedestrians to a grid map.The pedestrian map can be effectively combined with sensor map to improve the obstacle avoidance success rate of reinforcement learning algorithms in pedestrian environments.Aiming at the problem of a single strategy in the scene,we introduce a variety of different pedestrian strategies to train the navigation algorithm,which improves the navigation algorithm’s ability to deal with pedestrians with different strategies.In this thesis,we deploy the navigation algorithm trained in the simulation to the robot,and deploy the corresponding scene in the real environment,simulating static scene,dynamic scene,multi-robot scene and pedestrian scene.The robot can successfully complete the navigation tasks in several scenarios,which shows that the algorithm in this thesis is easy to deploy in the real environment.Finally,in order to make up for the limited observation range of the reinforcement learning navigation algorithm and improve the usability of the navigation algorithm,this thesis combines the above research results,traditional navigation algorithms and SLAM systems with task planning modules,and deploys them in practical environments to solve more complex mobile robot navigation tasks.