Research On Autoencoder And Intrinsic Curiosity Module Based Robot Grasping And Sorting Skills

Autonomous grasping and sorting operation is the key to the realization of intelligent home robots.Deep reinforcement learning with self-learning ability has huge advantages in robot operation tasks in unstructured home environments.However,in practical applications,there are problems such as the too-large sample size required by the algorithm and too long training time,which causes convergence difficulties and high training costs.Based on the auto-encoding and curiosity model,combined with the deep reinforcement learning algorithm,this article studies the autonomous grasping and sorting method of the robotic arm in the home environment.First of all,this article analyzes the existing reinforcement learning of the AC framework and determines the SAC as the decision-maker for the high-dimensional state and sparse rewards caused by the low efficiency of the reinforcement learning sample.Aiming at the problem that it is difficult to extract effective information from high-dimensional state input in reinforcement learning,a convolutional autoencoding feature extraction network combined with AC reinforcement learning is designed,which can perform dimensionality reduction and feature extraction on highdimensional pixel information.Then,to solve the problem of excessive exploration space caused by sparse rewards,a curiosity module and an internal reward network based on the reverse dynamic model are designed to enable the agent to increase exploration in a direction.Finally,the overall structure of the SAC-AE-ICM reinforcement learning algorithm for the sparse reward environment is established,and simulation experiments are carried out on the three reward sparsity tasks of the reinforcement learning test platform Open AI Gym.The results show that the algorithm is more effective than the original algorithm.Improved the difficulty of algorithm convergence caused by high-dimensional state input and sparse rewards.Secondly,this article is oriented to the requirements and characteristics of grasping and sorting tasks in the home environment,based on Mu Jo Co to establish a robotic arm simulation training platform,and design the network of the SAC-AEICM algorithm,aiming at the problem of slow convergence of grasping and difficult item sorting,based on the curriculum learning and variable internal rewards,the process of grasping strategy and segmented training is established.Based on the grasping algorithm,the target detection algorithm based on YOLOv3 is integrated,and the sorting strategy is designed and established.Experiments were carried out in the Mu Jo Co grasping and sorting simulation environment.The simulation results show that compared with the original algorithm,the problem of convergence difficulties is improved and the convergence speed is increased by about 25%.The success rate of grasping known objects and generalization of unknown objects is improved and the subsequent sorting process can be realized in the simulation.Finally,in this article,for the actual robotic arm grasping and sorting experiment,based on the ROS framework,an experimental platform of the UR5 robotic arm combined with the Realsense depth camera and the Robotiq two-finger gripper is established.Based on transfer learning,a pre-training scenario is constructed to reduce the number of actual robot training steps.Based on several common desktop object designs,a variety of scene capture experiments and sorting experiments have been carried out.The experimental results show that the convergence speed is increased by 43% compared with the non-migrated algorithm,and the SAC-AE-ICM algorithm is effective for regular objects on actual robots.The grasping success rate is about to 86%,and the grasping generalization of irregular objects is better.The designed sorting process can sort the grasped objects according to the preset shape.