Research On Path Planning Of Mobile Robot Based On Q Learning Algorithm

The mobile robot lacks the related prior knowledge in dynamic or unknown static environment, so that strong flexibility and adaptability are required to deal with various situations. First, for the mobile robot in static environment, this essay studies and designs a path planning algorithm based on the combination of deep auto-encoder and Q learning algorithm. Secondly, for the mobile robot in dynamic environment, this essay designs a two level planning strategy, and applies the genetic algorithm and Q learning algorithm in the path planning.The main research works and results obtained in this essay are stated as follow:(1) In static environment path planning, a data processing method is proposed, which combines the deep auto-encoder with Q learning algorithm. The BP neural network is used to fit the environmental feature data and the position data of the mobile robot, which can realize the nonlinear fitting of the robot's global coordinates and the environmental features. Then the position information obtained from BP neural network is used to get the reward value R as the feedback to the Q learning algorithm for Q value iteration. In the Q value iteration process, the robot constantly chooses different directions to move making the Q values optimal, which also means making the planned path optimal, so that the mobile robot realize autonomous learning. Finally, simulation experiment is implemented by MATLAB, and the experimental results show that this method can improve the ability of Q learning algorithm to deal with large scale data.(2) In dynamic environment, a two level planning strategy is proposed for path planning, in the method different planning strategies are adopted for different moving objects. The first layer uses a genetic algorithm to run the global path planning from the start point to the target point avoiding the static obstacles in the environment. After the first layer, the second layer strategy realizes dynamic obstacles avoidance by Q learning algorithm. When the mobile robot and the moving obstacles reach a safe distance, the robot returns to the global path to get to the target point. Finally, simulation experiment is implemented in the V-REP robot simulation platform. The experimental results show that using the two layer path planning strategy proposed in this essay can make the mobile robot avoid all moving obstacles and arrived at target point.(3) A new method of Q value table designing is proposed, which can solve the problem of storage space shortage and dimension disaster while the Q learning algorithm applied in dynamic continuous environment. In this method, the time is discretized into time intervals, and the Q value table is established by “time-action”. So, the problem of selecting an action according to the state is converted to the problem of choosing an action according to the time. The simulation results show that the proposed Q value table designing method is feasible, so the Q learning algorithm can be applied in the dynamic continuous environment.First, the data processing method which combines the deep auto-encoder with Q learning algorithm can improve the ability of Q learning algorithm to deal with large scale data. Secondly, the Q value table designing method provides a new idea for the study of Q learning algorithm applied in dynamic continuous environment. In addition, this essay provides a theoretical basis and experimental reference for the future study of intelligent robot's self-learning ability by using the Q learning algorithm.