Robotic Arm Grasping Policy Based On Deep Reinforcement Learning

In recent years,intelligent control algorithms have become increasingly attractive to researchers in the field of robot control,and deep reinforcement learning,as a branch of artificial intelligence,has been successfully applied to many aspects of robot control.With the application of intelligent control algorithm,robots can obtain the ability of selflearning and interaction with environment,which the traditional control algorithms cannot give.As an important actuator of the robot,the research on the intelligent control algorithm of the robotic arm has gradually attracted widespread attention.However,due to the complicated operation model of the robotic arm and the dynamic changes of the environment,the application of the algorithm in the robotic arm control still has many difficulties.Therefore,based on the method of deep reinforcement learning,this paper implements the complete control process of the robot arm from visual perception to action-decision,and proposes some solutions to improve the learning performance of the intelligent control algorithm in the field of robotic arm control.Firstly,based on the DDPG algorithm in deep reinforcement learning,we propose a modified DDPG algorithm(MDDPG)to solve the problem of low sample-efficiency and sparse rewards in deep reinforcement learning,according to the manipulation environment of the robotic arm in real world.The MDDPG algorithm consists of improvements of target adaptive and reward shaping.Then the MDDPG algorithm and the DDPG algorithm are trained separately in the simulation environment.The results show that the improved MDDPG algorithm can increase the learning speed by about 2 times.Secondly,considering the application conditions of the robotic arm in real world,computer vision methods are introduced into the control policy,and the multi-layer slow feature analysis(SFA)method is used to extract the key information in images.As for the problem of sparse rewards in the manipulation environment of the robotic arm in real world,a reinforcement learning algorithm based on intrinsic reward(IMAC)is proposed.Through the prediction of the state,internal incentive rewards are obtained,which are combined with external rewards to reduce the impact of sparse rewards and jointly optimize the control policy.The SFA-IMAC algorithm is trained in the simulation environment to verify that it has significantly improved the success rate of grasping;Then,as for the problems in transferring the control policy from the virtual environment to the real world,an adaptive control policy based on dynamic model is proposed.By introducing dynamic parameters and randomizing them,the adaptability of the control policy to the dynamic environment is improved.The adaptive ability is more conducive to the application of the control algorithm in the real world.Finally,all the algorithms are implemented on a 6-degree-of-freedom robotic arm experimental platform to verify and evaluate the performance.By pre-training the control algorithm in a virtual environment,and then transferring to the real arm,the experimental results show that the success rate of the robotic arm grasping in real world can reach 89%,which verifies the effectiveness of the algorithms and the improvements of the performance in the robotic arm control.This thesis provides a basic framework of intelligent control for future researchers,and contributes to the application of deep reinforcement learning in the field of robot control.