Deep Reinforcement Learning Based Object Grasping Of Dual-Arm Robot

With the development of robotics,the collaborative robot has been matured and had a profound impact on the industrial.Compared with one robotic manipulator system.the dual-arm robot has stronger efficiency to complete tasks or solve problems of complex targets.Therefore,the field of the dual-arm robot has gradually attracted researchers' attention.As the groundwork of the dual-arm robot's safety,collaborative control mainly refers to collision avoidance and coordinated motion between two arms.This research involves multi-agent cooperation,motion control,and target perception,etc.In order to prevent the collision of the two arms,the respective control strategy of a dual-arm robot needs to avoid competition with another during motion planning.The traditional method is to establish a precise mathematical model for the manipulator and plan the motion trajectory of the end effector according to the features of various tasks,then calculate out the angle of each manipulator's joint by the inverse kinematics equation.However,this theory of manipulator lacks generalization and requires a lot of computing resources and time.Recently,the multi-asent cooperation algorithm has provided a solution for the controller of manipulators.The agent using deep reinforcement learning can explore the action space autonomously.reduce the competition between each other,and increase the overall coordination.Therefore,this thesis aims to utilize the DRL(deep reinforcement learning.DRL)mechanism to perform the coordinated motion of the two manipulators and improve the versatility and robustness of the dual-arm robot.The research is mainly divided into simulation and ph,ysical experiment.In the part of the simulation,an algorithm named DADDPG(Dual-Arm Deep Deterministic Policy Gradient.DADDPG)is proposed,which assigns a control strategy to each of the two manipulators and shares their observations and actions with each other.The dual-arm robot is trained by a method of "rewarding cooperation and punishing competition"learning how to complete the cooperative tasks.The effectiveness of the algorithm is verified in the MuJoCo(Multi-Joint dynamics with Contact,MuJoCo)simulation environment built in this study.For the physical experiment part,an ROS(Robot Operating System,ROS)based dual-arm robot control architecture is designed,by using a Kinect camera and Mask-RCNN(Mask Region Convolutional Neural Network,Mask-RCNN)algorithm for target positioning,so that the robot could successfully grab a thin stick.This thesis firstly introduces the research significance of dual-arm robot,expounds the research purpose of this topic,reviews the progress of DRL and dual-arm robot in the domestic and international,then explains some basic concepts and classification of deep reinforcement learning algorithms.Secondly,two kinds of algorithms in DRL are discussed in detail:DQN(Deep Q Network)and DDPG(Deep Deterministic Policy Gradient,DDPG).The characteristics and application fields of the algorithms are compared.Furthermore,the optimization of DRL in practical application is emphatically analyzed.Thirdly,the problems of sparse reward and multi-agent coordination in DRL are presented.Then,inspired by the HER(Hindsight Experience Replay,HER)and Multi-Agent DDPG.the DADDPG algorithm is proposed to guide the movement of the dual-arm robot.Then,chapter 4 describes the progress of creating three MuJoCo simulation experiment environments and introduces how to use DRL for training agent.After trained,the virtual robots are tested for the algorithm in MuJoCo.Finally,this thesis demonstrates the implementation process of using this DRL algorithm to control the entity dual-arm robot via ROS.This study deploys a distributed architecture of Kinect,two UR3 robots.DRL and Mask-RCNN algorithm,which reduces the coupling between modules and makes the physical dual-arm robot successfully complete the coordination task of objects grasp.