Research On 3D Classification Of Point Cloud Based On Deep Learning For Sorting Technology

Recently,with the development of artificial intelligence and "Made in China 2025",a large number of industrial robots have been applied to the sorting system,and the robot-led production mode has replaced the traditional manpower mode,reducing the damage of the harsh environment to the human body and greatly promoting the degree of automation,work efficiency,and reduction of production costs of the task.However,the 2D images lacks 3D geometric information and is obviously affected by texture,light and so on,which results in a visually blurred image and a low overall quality.At the same time,the traditional 3D point cloud visual processing is complicated and computationally complex.Although the emergence of 3D deep learning networks provides new ideas,the manual labeling of datasets often consumes a lot of manpower and material resources.Therefore,this thesis adopts the optimized deep learning framework of end-to-end 3D point cloud processing to reduces the above effects and enhance the learning of local feature information,thereby improving the accuracy of recognition results and the convenience of transplantation.In addition,this thesis uses the datasets generated by the virtual environment to train the network and build an industrial robot classification and sorting system for verification in the real environment.First,the platform of the robot sorting system was designed.The 3D vision is used in this thesis,which is different from the method of robot programming and teaching.The method of this thesis needs to use virtual environment to make datasets,needs to use camera to acquire object information and needs to process the corresponding data.Therefore,this thesis designs the platform and scheme of the robot sorting system,and elaborates on the selection of each hardware in the system and the technical principle of the software.Second,the deep learning framework for end-to-end 3D point cloud processing is optimized.The ResNet3D network is proposed in this thesis,which uses the R3Block module to break the gap between the 3D deep learning network and the classic network,and better acquire local features.In this thesis,the shape classification datasets named ModelNet40 and the scene segmentation dataset named ScanNet,the basic theory of deep learning in point cloud,and the proposed ResNet3D network are elaborated.It also proves the validity of the network on the public datasets,and provides theoretical and experimental basis for the classification and segmentation techniques in the sorting task.Then,visual calibration of the robot sorting system is achieved.This thesis uses an eyeto-hand installation between the Kinect v2 depth camera and the UR5 robot.This thesis first elaborates on the principles of depth camera imaging,internal and external reference calibration,distortion,hand-eye calibration and so on,and then uses the "Zhang Zhengyou calibration"method to calibrate the Kinect v2 to obtain the calibration error and the camera's own parameters.The VISP library performs the Eye-to-hand calibration experiment to obtain the conversion relationship between the Kinect v2 depth camera coordinate system and the UR5 robot coordinate system.Finally,a robot sorting system based on deep learning 3D point cloud classification is built.In this thesis,the angle of the virtual camera is adjusted in the Blensor virtual environment to obtain a large number of pure virtual datasets with labels.In the real scene,the point cloud data on the surface of the target object is collected and processed by PCL,such as removing NaN points,extracting cuboid point cloud data,removing planes and deleting points from outliers.In addition,virtual data sets and real data also require processing such as unit ball normalization and the FPS.In this thesis,the 3D classification sorting network model is trained by using the created virtual datasets,and the surface data of the target object collected in the real environment is imported into the trained model to guide the UR5 robot to grab the target object.The experimental results verify the feasibility of the method in real scenes.