Robot Skill Acquisition Based On Imitation Learning And Reinforcement Learning

Inspired by the two learning stages of humans' growing up process,one is learning from teachers and the other is exploring new knowledge autonomously.Researchers began to apply imitation learning(IL)and reinforcement learning(RL)to robot skill acquisition.Especially with the advent of depth cameras and stereo vision sensors,robot can obtained more abundant information for policy learning and interactive exploration.Therefore,in this paper robot skill acquisition based on imitation learning and reinforcement learning is carried out.The research contents include the following aspects:Firstly,the RGB-D image interaction demonstration(RGBD-ID)is proposed based on the property of RGB-D camera which can reflect the three-dimensional features.It is a task-oriented and intelligent-interaction-based method.In this paper,Kinect V2 is used for the object recognition and location,and MoveIt! is used for high-level action planning.We take the RGB-D image as our demonstration space.An object and an action were selected in the RGB-D image to guide the robot to operate the corresponding object in the real workspace.Multiple interactions form the demonstration trajectory of a motion skill.Secondly,Learning from demonstration(LfD)has been studied.A skill learning model(OPLN)consisting of objects list network(OLN)and policy learning(PLN)is proposed according to the humanoid high-level behavior of skill acquisition and the characteristic that operator interacts with only one object and one action in RGBD-ID.OLN and PLN are constructed respectively with LSTM neural network.OLN reflects orders relationship between the objects while PLN reflects positional relationship,so that robots can achieve independent inference and skill acquisition at a high cognitive level.Thirdly,this paper pays attention to the robot skill acquisition based on reinforcement learning.The robot can learn policy autonomously by interacting with environment.Aiming at the robot operation task,a reward function design method based on objects configuration matching is proposed.The similarity between the objects target configuration and the current configuration is calculated according to the vector similarity measure method,and then an instant reward function for the similarity is constructed.The Actor-Critic algorithm is used as the main structure to construct the robot reinforcement learning model.The model combines with the reward function to learn and optimize the skill policy.Finally,the experimental platform is built for the above methods.The hardware system of the experimental platform includes UR5 robot,Robotiq 2-finger gripper,Kinect V2 depth camera and so on.The software system consists of ROS operation system,MoveIt! motion planning library,Matlab,pytorch neural network framework and so on.The Block Stacking task and the Pick and Place task were setup to verify the effectiveness and feasibility of our three methods for the robot skill acquisition.