Research On Intensive Learning Based On Motivation And Its Application

Traditional reinforcement learning(RL) mainly focuses on the extrinsic motivation, which designs special external reward signals for a specific and concrete task, to drive the agent to learn a behavioral strategy that maximizes its long-term cumulative rewards. The reward signals usually need to be well designed according to the characteristics of the scenario and the agent to improve the learning performance, and thus lack designing commonality and learning initiative.In this paper, aiming at the need for reducing the complexity of reward designs and for implementing agents’ autonomous learning, a quantitative emotion-based motivation model is proposed by mapping agents’ sensed states into emotional dimensions, which is inspired by the role of emotions in human decision-making. The emotional motivation is used as a supplementary reward on the basis of the external reward or used as the only reward, thus forms a loop-locked "perceive-evaluation-intrinsic emotional reward and external reward-RL algorithm-actionperceive" emotion-motivated RL framework. The intrinsic emotional motivation is taskindependent and thus has a certain university. In the quantitative model, the curiosity is used to control agents’ exploration preferences for strange or familiar states by evaluating the novelty of the state-action pair; the controlling desire is used to adjust the trade-off between "conservative" and "radical" by evaluating agents’ control power over the environment model; the happiness index is used to adjust the external reward by evaluating the current state-action’s relative happiness level; and combined together to realize the adjustment of agents’ learning preferences and behavioral patterns.On the basis of the proposed emotion-motivated RL framework, the methods of using intrinsic emotional motivations to accelerate the learning of concrete tasks are talked in detail. The first method is to use the intrinsic emotional motivations to explore the uncertain environment and learn the environment transitioning model ahead of time for later use, which also alleviates the "exploration-and-exploitation" dilemma in traditional RL; while the second method is to combine the intrinsic emotional motivation with the external reward as an ultimate reward function, directly driving agents’ learning more effectively. Through the simulation experiments in the cat catching mouse scenario on Robot Operating System(ROS), both methods have shown relatively good results compared with pure external reward-motivated learning, and prove the effectiveness of using emotions as the intrinsic motivation in the aspect of accelerating concrete tasks’ learning and the rationality of the proposed quantitative emotion-based motivation model.In addition, using emotion-motivated RL to personalize agents is also discussed in this paper. By adjusting corresponding parameters in the emotion-based motivation model and introducing emotional dimensions of higher abstraction level, it would be easy to design various kinds of agents with different learning preferences and behavioral patterns, which could be used in scenarios like human-machine interactions, virtual character designing and so on.