Research On Motion Control Of Mobile Robots Based On Reinforcement Learning

In recent years, reinforcement learning(RL) has been one of the key research areas in artificial intelligence and machine learning. Reinforcement learning is different from supervised learning in that teacher signals are not necessary and a reinforcement learning system learns by interacting with the environment to maximize the evaluative feedback from the environment. Thus, reinforcement learning methods have wide application areas in solving complex optimization and decision problems, where teacher signals are hard to be obtained.As one of the key techniques in mobile robots, the aim of motion control is to regulate the velocity and direction of the mobile robot and keep the robot's position and attitude consistent with the planned trajectory. Due to the complexity of external environments and the uncertainty in robot dynamics, motion control of mobile robots is still a difficult and hot topic. In this dissertation, the path following control problem in mobile robot motion control is studied and the main focus is to apply reinforcement learning methods for optimization of motion controllers of wheeled mobile robots. The performance of the proposed methods is evaluated and verified in experimental platforms of mobile robots. The main research work completed in this thesis includes the following aspects:(1) The approximate policy iteration methods in RL are studied in detail and a novel correlation analysis method is proposed to select the most appropriate basis function for the least-squares policy iteration (LSPI) algorithm . It has been illustrated that based on the correlation analysis of polynomial basis functions, the generalization ability of LSPI can be improved.(2) A novel self-learning path-following control method based on reinforcement learning is proposed for a class of wheeled mobile robots. In the proposed method, the path-following control problem of mobile robots is modeled as a Markov decision process (MDP) and by using the least-squares policy iteration (LSPI) algorithm and the kernel least-squares policy iteration (KLSPI) algorithm, the lateral control performance of the two-wheeled mobile robot can be optimized in a self-learning style. The KLSPI algorithm uses kernel methods with automatic feature selection and value function approximation in policy evaluation so that better generalization performance and learning efficiency can be obtained. (3) By making use of the Pioneer3-AT mobile robot platform, experimental studies are conducted to evaluate the performance of the path-following control method based on RL.The sampled motion data from real mobile robots are used as training samples of RL and the approximate policy iteration algorithm is adopted to learn a policy with optimized performance. Then the automatic design of the lateral motion controller is realized. The efficiency of the proposed method is verified by experimental results.(4) The application of reinforcement learning in multi-robots formation control is studied. A learning control method is suggested to keep the formation control of multi-robots based on reinforcement learning. The parameters of l-φcontrol , which is used to realize the design of the controller, are optimized by the approximate policy iteration algorithm. Some initial simulation and experimental results have been obtained.The research work in this thesis not only analyzes and makes some improvements for basis function selection in reinforcement learning algorithms but also is beneficial to apply reinforcement learning in uncertain optimization problems including the controller optimization of mobile robots.