Robotic Behavior Control Based On Reinforcement Learning

In recent years,the research and application of robotics has received unprecedented attention globally,and various countries have successively introduced national development strategies related to robotics.China has also released the "Robot Industry Development Plan(2016-2020)",striving to achieve leap-forward breakthroughs in the robot industry with great potential,and make the development of robot technology an effective driving force to achieve the goal of "Made in China 2025".In the complex and dynamic environment where robots are located,how to design effective controllers and decision-making mechanisms to make robots work stably in complex and unknown environments and adaptively complete tasks is an important issue in robot research.By using the sample data collected by interaction with the real world,reinforcement learning(RL)has become an effective method to achieve optimal control of a robot.Although RL has progressed in many areas,it is still difficult to apply it in robotic environments.The most important reasons are threefold:(1)The state and action space of the robot are generally high-dimensional and continuous.This huge decision-making space makes ordinary RL methods,even with dimensionality reduction and approximation,quickly fail.(2)RL requires a large number of samples to explore the high-dimensional continuous state space of the robot.There is often a correlation between the internal structures of robots,and this correlation changes dynamically in time and environment.However,the RL algorithm cannot accurately perceive the dynamic of this correlation,so the algorithm often encounters the problem of low learning efficiency.Moreover,the RL algorithm is often not interpretable,and it is completely unable to explain the reason for the success or failure of the strategy.(3)RL completely learns from scratch by trial and error,and cannot integrate human experience and knowledge.Real robots need to interact with humans.If human knowledge cannot be integrated,it will lead to unexpected robot behavior that even accidentally injures humans.At present,although there are many ways to solve the problem of integrating human prior knowledge,the advantages and disadvantages of these algorithms are not clear.Therefore,an adaptive human-agent collaboration algorithm is needed.In order to alleviate the above issues,this thesis will discuss the following three aspects: decentralized multi-agent RL for robot control,topology-aware robot control learning based on attention mechanisms,adaptive learning via human-agent collaborative interactions.Detailed experimental comparison and case analysis of the proposed method against the original methods on multiple benchmark robot environments prove that benefits of the proposed methods in addressing the above three challenging issues in robotic RL.