Deep Reinforcing Learning With Self-introspection Mechanism On Multi-scale Reward

The combination of deep learning and reinforcement learning algorithms,i.e.,deep reinforcement learning(DRL),has made a lot of important breakthroughs in recent years,especially in complex tasks for dynamic systems.However,real world applications of reinforcement learning must specify the goal of the task using a manually programmed reward function,which limits the performance of the learning systems.In this paper,we propose a potential-based reward shaping method for DRL inspired by biological and psychological research.First,the introspection-reward is formulated to describe whether the current action is better or not compared with the past action sequence of the previous states.Then,an introspection-agent is constructed to correct the updating of the main agent,which establishes a new learning framework that allowing the agent to continuously reflect and compare its past experience in the learning process like human beings.The experimental results on the classic Atari games and the Super Mario games demonstrate that the DRL algorithm with introspection-reward can achieve better performance.In addtion,the proposed method is easy to implement and to be extended to the existing algorithms.