Reinforcement Learning Based Mobile Robot Autonomous Navigation

Mobile robot have the ability of moving in the environment it is in,it can finish the navigation mission without human' intervene.Recently,reinforcement learning is using more and more widely in the field of mobile robot navigation,so as to improve the ability of adapting to continually changing environment.Focusing on the problem of dimension explode,people use the method of functional approximation to construct the relationship between the state space and the action space.Neural network,such as BP neural network and RFB neural network,is widely used as it can represent the relationship very well.However,all these methods have a disadvantage of needing prior knowledge and lack of autonomy.In this thesis,we combine the ART2 neural network to navigation algorithm,realized a mobile robot navigating method based on reinforcement learning,the content are as follow:(1)The existing problems of autonomous navigation for mobile robot based on reinforcement learning are analyzed.The autonomous navigation of mobile robot based on Q-learning algorithm is simulated.For the discretization of continuous state space,block method and fuzzy logic discretization method are designed respectively,and through simulation experiments,we compare and analyze the shortcomings of the two methods,such as poor autonomy,complex classification and slow convergence speed of the algorithm(2)Designed a ART2 based navigating algorithm combining reinforcement learning,realized the mobile robot online learning by combining the ART2 neural network's character of memory and the autonomy of the reinforcement learning.Using the competitive learning mechanism of the ART2 neural network to generate disperse state space without man's appointment,the robot will itself partition the environment character and improve the robot's performance.This thesis solved the problem of the dimension explode meanwhile improved the autonomy.Import eligibility trace into the Q-learning algorithm to improve the learning speed.(3)Designed a simulation experiment based on Mobotsim software.Verified the effectiveness of the proposed method using in mobile robot navigation.Meanwhile,do a comparative experiment between the Q-learning with and without eligibility trace to show the speed's improvement during the learning process.(4)Use the crawler robots to set up mobile robot autonomous navigation experiment system.The system uses ultrasonic distance measurement module and visual system for environmental identification.Developed a mobile robot control system based on embedded system.Completed the ultrasonic ranging data acquisition and V4L2 based image acquisition.Transplanted Qt library to the development board and control software development to achieve a remote monitoring function based on Qt.Achieved the development board on the robot chassis control function.The validity of the proposed method is verified on the platform...