Multi-Granular Reinforcement Learning Via Knowledge Transfer

Reinforcement learning is an effective method to solve decision-making and control problems.However,it is highly dependent on data,long learning time and prone to dimension disaster.How to improve the efficiency of reinforcement learning becomes the key.Multi-granular model description method is agent centered,which is an information representation method for modeling environment and objects.Transfer learning can use learning experience of one or more source domain tasks to solve new tasks through learning previous feature information and data experience.Because of the uncertainty of the environment,the simple transfer reinforcement learning may have negative transfer and other problems.Therefore,based on reinforcement learning,multi-granular model and transfer learning research,this paper proposes a multigranular reinforcement learning method based on transfer learning.In multi-granular reinforcement learning,the environmental model is granulated to improve the recognition and understanding of the environment by reinforcement learning and transfer learning algorithm,avoid dimension disaster and negative migration,so as to improve the efficiency of transfer reinforcement learning.In the process of learning,multi-granular model is used as the object and carrier of knowledge transfer,and a bridge of learning transfer is built between different granularity,and a multi-granular transfer reinforcement learning algorithm is designed.In addition,for the proposed algorithm,the maze problem and the inverted pendulum problem are used to carry out a number of experimental simulations,and the navigation control experiment of the mobile robot is completed to verify the effectiveness of the research method.This paper includes:(1)multi-granular model is defined,and multi-granular reinforcement learning algorithm based on transfer learning is proposed systematically,including transfer from fine-grained to coarse-grained,from coarse-grained to finegrained,and transfer between different granularity of similar tasks.The principle and process of the algorithm are explained in detail;(2)for typical applications,namely grid maze problem and inverted pendulum problem Numerical simulation experiments and real system experiments are designed to verify the effectiveness of the multi-granular reinforcement learning algorithm based on transfer learning,and the experimental results are analyzed and discussed.The related results have important reference value and practical significance for the research of reinforcement learning algorithm and typical application.