Study Of Robot Manipulation Skills Learning Algorithm Based On Demonstration Data

With the development of science and technology,the robot industry is growing rapidly.At the same time,the requirements for the intelligent robots continue increasing.Traditional modeling and control methods can only make the robot skilled in a single structured environment and master invariable manipulation skills.When confronting the complicated environment,Deep Reinforcement Learning(DRL)algorithms can make robots learn more flexible manipulation skills.However,when implementing DRL to the learning of robot manipulation skills,the performance is often not so good due to the low sampling efficiency,difficult reward shaping,and unstable convergence.In order to overcome these difficulties,this thesis carries out the following research by introducing demonstration data.Firstly,a linear space mapping tele-operation system is built to effectly collect demonstration data.After eslablishing the master and slave kinematics,operators can remotely control the slave robot to complete the manipulation.The feasibility verification experiments are carried out on both the simulation environment and the physical robot.And the linear space mapping tele-operation system provides a demonstration data set for the subsequent modified DRL algorithms.Secondly,this thesis proposes the Demonstrations Initialization Hindsight Experience Replay(DIHER)algorithm to overcome the low sampling effciency.The DIHER algorithm is built within the framework of distributed Deep Deterministic Policy Gradient(DDPG),and its key trick is to initialize the experience replay buffer with a certain amount of demonstration data in the initial training stage.The experimental results show that the addition of demonstration data can effectively accelerate the learning of robot manipulation skills.The more demonstration data,the faster learning speed.Besides,the DIHER algorithm can also ensure the stability of the manipulation skills.Thirdly,in order to overcome the hard reward shaping and the unstable convergence problems,this thesis proposes the Goal-oriented Generative Adversarial Imitation Learning(GGAIL)algorithm.The GGAIL algorithm consists of multiple tricks,which includes goal-oritend discriminator,relabeling expert data,annealing weight and early stop of the discriminator training.The experimental results show that the annealing weight and the early stop of the discriminator training are essential tricks for its effect.Compared with the DIHER algorithm,the GGAIL algorithm is more robust to the noise perturbation.In summary,the proposed algorithms can effectively realize the learning of robot manipulation skills.And it is helpful in improving the intelligent level of robots.