Object Focused Reinforcement Learning Algorithm Research

Reinforcement learning is an important branch in the field of machine learning,which adopts a basic idea in the theory of learning and intelligence—“learning from interaction” to solve sequential decision problems.Due to its unique learning mechanism—trial-and-error search,reinforcement learning has gained increasing attention from researchers in recent years.This paper attempts to study the reinforcement learning algorithms from the perspectives of improving algorithm generality and learning efficiency.The main contributions are as follows:Firstly,a approach of measuring distance between Markov Decision Processes(MDPs)is discussed.It completes measurement of MDP similarity especially in different state space or state representatives,thus generalizing algorithm.Secondly,based on the traditional object-focused q-learning(OF-Q)algorithm,a simplified object focused q-learning algorithm(SOF-Q)is discussed,which uses a new control strategy to avoid the risk of ignoring parts of the state space.Compared with OF-Q,this new method can double the running speed,cut down the memory consumption by half and obtain more stable performance.Finally,object focused Q-learning algorithm based on dyna architecture is discussed,which estimates the global Q-values by combining dyna architecture and object focused MDPs.Experimental results show that the proposed method can solve the illusion of control which exists in OF-Q and SOF-Q efficiently,thus improving the robustness of algorithm.