| Reinforcement learning algorithm is used on intelligent navigation based on analyzing of mobile robot navigation control and comparability of reactive navigation and reinforcement learning model, Q-learning algorithm based navigation knowledge acquisition is mainly studied.Temporal Difference (TD) algorithm, Adaptive Heuristic Critic (AHC) algorithm and Q-learning algorithm that are main reinforcement learning algorithm are researched. The key problems in reinforcement learning algorithm—tradeoff of exploration and exploitation, continuous state and action, credit assignment, partially observable Markov decision processes are analyzed, and some solutions are presented. Distributed reinforcement learning is introduced. The brief algorithms of distributed reinforcement learning are presented; the problems and their solutions are discussed. The action chosen strategy in standard Q-learning algorithm is greedy, i.e., exploitation, which tends to fall in local optimization. Some solutions are presented, but blind exploration and repeatedly learning after finding optimal path exist. An improved Q-learning algorithm based on exploration region expansion is proposed to avoid the local optimization and blind exploration. Meanwhile, other feasible path is sought where agent encounters obstacles, which makes the implementation of the algorithm on real robot easy. An automatic termination condition is also put forward, therefore, the redundant learning after finding optimal path is avoided, and the time of learning is reduced. The validity of the algorithm is proved by simulation experiments.The generalization algorithms of reinforcement learning are analyzed aiming at continuous state and action. However, the previous solutions have the problems about continuous action, neural network based continuous state and action Q-learning is proposed. The dimension disaster of reinforcement learning is resolved; it makes the implement of reinforcement learning on robot possible.
|
PDF Full Text Request