Navigation Control For Autonomous Mobile Robots Based On Reinforcement Learning

Autonomous mobile robot can work in the complex and unstructured environment without human intervention. It has a high degree of self-planning and adaptability in uncertain environments. The navigation technology is a key to the intelligence of mobile robots. Robots need to learn quickly in order to improve the ability of adapting unknown environments, and resolve complex problems without complete knowledge. Reinforcement learning algorithm provides the self-studying ability, and has been received extensive attention in the field of the navigation of autonomous mobile robot systems.Reinforcement learning imitates the process of animal learning, and seeks the optimal mapping from states to actions based on trial and error method. Traditional learning algorithms always have several problems, such as slow convergence, time constraints when estimating delayed reward, memory-consuming and time-consuming during calculation. In this thesis, relative value iteration reinforcement learning (RVI- RL) algorithm is proposed with reasonable deformation, which is based on the traditional theory of reinforcement learning, and combined with the relative value iterarion theory and the optimization theory. This algorithm is based on the Markov decision process (MDP) environments, and does not need to estimate the average reward.RVI-RL algorithm is a model-free algorithm, which not only eliminates the estimation of the average reward in the task, but also eliminates the bias due to the constant modification of estimation error of the average reward constantly. The classical taxi problem is taken as our experiment model. The environment is initialized as a grid map. However, the information is unknown to the taxi. The taxi drives autonomously through trial and error techniques to obtain the state information of the environment, and ultimately gets the optimal mapping from states to the actions.Experiments show that RVI-RL algorithm could converges much faster and stable than R-learning algorithm and Q-learning algorithm to the optimal strategy in the navigation of autonomous mobile robots. The new algorithm shows the ability of inducing cooperation between two robots, which is more appropriate than the traditional Q-learning algorithm in the collaborative planning problem. Although RVI-RL algorithm eliminates the estimation of the average reward, there still exists "Dimension Disaster" problem in the large state space. This thesis also discusses the feasibility of introducing the stratification into the RVI-RL algorithm, and experiments show the better convergence to the optimal strategic than MAXQ algorithm.