Research Of Robotic Grasp For Stacking Algorithm Based On Deep Reinforcement Learning

For the sake of adapting to the complex and changing environment in the real world,intelligent robots need to perceive the environment and react,and then complete specific tasks through multiple actions.In industrial production scenarios,most of the existing robotic arm systems perform actions based on manually set rules.They can only pick up specific objects at a fixed position and place them in a previously defined position.This makes it impossible to choose a suitable placement according to the shape and the size of the object,and can only grasp a single shaped object and put it at a fixed position.In the field of academic research,many researches relating to the operation of the robotic arm only focus on the action of grasping,and ignore the operations that should be performed after grasping,such as placing the grasped object on another platform.In order to solve the above problems,this thesis proposes a stack-oriented grasping task learning method,which grasps objects for appropriately placing them by learning actions from visual observations in an end-to-end manner.Due to the rapid development of deep reinforcement learning in recent years,this thesis introduces it into the designed system.Using the decision-making ability provided by deep reinforcement learning,this system understands the environmentthrough the pictures taken by the camera and learns to make a better decision by trial and error.In this thesis,the model-free deep Q learning method is used to learn the grasping for stacking strategy from scratch.Specifically,this thesis maps the images to the actions of the robotic arm through two deep networks:the grasping Network(GNet)using the observation of the desktop and the pile to infer the gripper's position and orientation for grasping,and the stacking network(SNet)using the observation of the platform to infer the optimal location when placing the grasped object.To make a long-range planning,the information of two observations is integrated in the grasping for stacking network(GSN).Consider that reinforcement learning is both sample inefficient and unstable,this thesis designed three auxiliary tasks to help the network extract task-related features,which accelerated the convergence process and improved the final performance.The grasping auxiliary task is the prediction of the number of the desk objects,and the stacking auxiliary task is heap height prediction task.The grasping for stacking auxiliary task is an object-centered feature learning task.This task helps the grasping for stacking network(GSN)to fuse information for making good decisions.In this thesis,we train and test the baseline method and conduct ablation experiment in the V-rep simulation environment,and then evaluate the performance of GSN and baseline method on the grasping-stacking task in a real scene.Finally,it is proved that the proposed grasping for stacking network(GSN)can handle the stacking tasks which have randomly placed objects with different sizes.