Research On Multi-object Sorting System Based On Deep Learning

In recent years,with the recovery of the industrial field,the annual increase in labor costs and the impact of the aging population,robots have become the mainstream trend in many fields in the future.As a commonly used technology for robots,robotic sorting has been widely used in household services,industrial manufacturing,logistics and warehousing and other fields.Robot sorting is a complex task,which mainly involves scene perception,target detection,pose estimation and grasping planning,etc.It has become a research hotspot for researchers.However,most of the robot sorting systems at present can only complete the sorting task for a single type of target object or multiple types of target objects separated from each other,and the sorting of multiple types of objects that are stacked or obscured in unstructured scenes has not been good.The solution.In order to enable the robot to perform safe,stable and accurate sorting operations on stacked objects in unstructured scenes,this paper introduces the deep learning method into the robot sorting system to obtain the target object category and grasping pose.Through the proposed sorting sequence reasoning algorithm,an orderly and autonomous sorting process is realized.This article mainly carried out the following research work:(1)Taking into account the requirements of complex unstructured scenes for robot automation and intelligence,a robot autonomous sorting system that integrates target object detection,scene instance segmentation,pose recognition and sorting sequence reasoning is built,and The main hardware devices in the system are introduced.(2)Aiming at the problem that traditional horizontal border-based target detection cannot effectively recognize the posture of the object when the target object has a large tilt in the image,this paper uses a refined single-stage rotating target detection network R3 Det to predict the type,position coordinates and tilt angle of the target object in the scene,so as to provide the basis for the subsequent grasping pose estimation and robot sorting.Through the constructed data set,the training and evaluation of the rotating target detection network is completed.(3)By replacing the mask head of the instance segmentation model,an instance segmentation network based on the optimized Mask R-CNN is built,which solves the problem of imprecise segmentation of the original network objects.Then the camera calibration is used to convert the segmented object surface pixel coordinates into a three-dimensional point cloud,and apply uniform sampling and outlier removal to the segmentation point cloud,thereby improving the speed of subsequent point cloud analysis.Then use PCA to predict the trend of the main normal on the surface of the object,and the Euler angle method is used to calculate the grasping pose of the target in combination with the rotation angle of the object.Finally,an object sorting order reasoning algorithm based on prior knowledge is proposed to solve the problem of robot damage or failure of grasping due to insufficient understanding of the scene.(4)An experimental platform for robot multi-object sorting was built,and the software design of the sorting system was carried out through the server/client architecture.The performance of the proposed robot sorting system has been comprehensively tested,including a single object grasping experiment,a multi-object sorting experiment of the same type,and a multi-type object sorting experiment.Experimental results show that the multi-object sorting system based on deep learning proposed in this paper can sort stacked objects autonomously,safely and stably in unstructured scenes.