Research On Key Technologies Of 3D Workpiece Recognition Based On Deep Learning

In the manufacturing industry,intelligent perception is an essential technology to achieve automation and intelligence.More than 80% of the outside information obtained by humans comes from vision.Therefore,visual perception is a very important part of intelligent perception,with detection and identification of industrial workpieces being the first needs in manufacturing.As the objects to be processed in the manufacturing industry gradually transfer from a two-dimensional representation to a three-dimensional representation,the low textures of the workpieces themselves and the small differences between the workpieces and the environment are more difficult to differentiate.These conditions also make it more difficult to detect and identify workpieces using traditional image processing and point cloud processing methods.In the industrial field,for complex three-dimensional shapes of the workpieces that cannot be handled by traditional algorithms,deep learning can independently learn the geometric features and image characteristics of the complex workpieces.This research can not only speed up the development of workpiece detection and recognition algorithms in the manufacturing industry,but it can also provide a new way of thinking and a method for the detection and identification of complex workpieces.Therefore,the key technologies of detecting and identifying low-texture three-dimensional workpieces are studied in this paper.The main research contents and innovation points are as follows:(1)In order to enhance the self-learning scale and pose invariance of the network,a two-dimensional network architecture Polish Net-2d for polishing workpiece detection is constructed by introducing a transformation network X-Net with self-learning function for scaling and translating workpiece objects into the backbone networks,such as Res Net101 and RPN.The results show that the scale and pose invariance are improved,and the trend is more obvious with the increase of iterations.Among them,X-Net can learn unsupervised regions extracted from the region proposal network on the training set,which improves the detection rate of Polish Net-2d,a two-dimensional workpiece detection network.From different perspectives such as learning rate and batch size,this paper makes comparative experiments based on eight datasets,such as single workpiece datasets and multi-workpiece datasets.The experimental results show that the global convergence of Polish Net-2d has high robustness.Experiments on test dataset show that Polish Net-2d proposed in this paper can make good use of different types of datasets,and the loss function can converge to an ideal value quickly.Because the backbone network of Polish Net-2d and Faster R-CNN are basically the same,the difference between the two detection networks is that Polish Net-2d introduces the transformation network X-Net,which enables unsupervised learning of the target transformation in the proposed areas obtained by the region proposal network,thus making the Polish Net-2d more robust in translation and scaling.(2)Aiming at dealing with the diversity of scale and pose of three-dimensional workpieces in practical application scenarios,the transformation network,R-Net with self-learning translation,rotation and scaling for three-dimensional workpiece are introduced in the backbone network,Point Net.By introducing the hierarchical feature extraction network,the corresponding three-dimensional workpiece features are extracted in different feature pyramid layers based on different scales.Polish Net-3d,a three-dimensional network architecture for polishing workpiece point cloud recognition,is constructed.Based on the three-dimensional workpiece point cloud datasets with different scales and pose diversity,the comparison experiments with R-Net and HFEN show that after adding R-Net and HFEN,the average classification accuracy of the network test is better.There has been a obvious improvement.In this paper,three optimization strategies,i.e.mixed training,batch training and batch normalization,are used to conduct comparative experiments based on three-dimensional workpiece datasets with different point cloud densities.The results show that Polish Net-3d has high robustness to point clouds with different densities,and can achieve high recognition rate even when point clouds are partly missing.Experiments on the three-dimensional point cloud dataset of polishing workpiece constructed in this paper show that the recognition rate of the workpiece's three-dimensional point clouds is very ideal because Polish Net-3d introduces the hierarchical feature extraction network,and the transformation network R-Net.Through point cloud data augmentation,Polish Net-3d achieves recognition accuracy of 0.9726 on test dataset.(3)Aiming at solving the problem that there are very few kinds and quantities of industrial workpiece datasets that can be directly used to train deep neural network,based on theoretical workpiece model,real environment background data and actual data collected with individual workpiece,and according to illumination transformation and multi-view multi-source image fusion algorithm,multi-view theoretical workpiece dataset,theoretical workpiece model-real environment background dataset and individual real workpiece – real environment background dataset are constructed.There are three types of multi-view datasets in real environment background.The dataset contains 1,298,640 images,including 1,155,000 images in the training set and 143,640 images in the testing dataset.The experimental results show that the dataset constructed by this method can satisfy the training requirement of deep neural network,and the trained network can be used in practical industrial occasions.Based on the three-dimensional model of theoretical workpiece,the point cloud dataset of three-dimensional workpiece is obtained through three processes: point cloud data format conversion,point cloud upsampling and point cloud data augmentation.After being processed by five point cloud data augmentation methods,a total of 25,200 point cloud data are generated from theoretical three-dimensional model and 37,500 point cloud data are collected from real scenes,and they are applied to the actual three-dimensional workpiece recognition experiments.The results of workpiece recognition experiments show that the method proposed in this paper can be used to automatically construct the three-dimensional workpiece dataset,which can be applied to situations with diverse scales,positions and poses.Aiming at solving the time-consuming problem of data annotation for training deep neural network,based on the two-dimensional workpiece dataset constructed in this paper,the format of two-dimensional test results annotation file is analyzed according to illumination transformation,thresholding and edge tracking algorithm,and the annotation file is automatically generated according to the corresponding format.Compared with the traditional manual labeling method,the proposed method can greatly reduce the labeling time and improve the labeling efficiency.The test results of Polish Net-2d show that the annotation files generated automatically by this method can meet the training and testing requirements of deep neural network.(4)Aiming at dealing with low texture of the workpiece surface and blurred motion in the collected images,based on the decision level fusion method of multi-source data,the edge fusion algorithm and the edge connection algorithm based on tangent vector,proposed in this paper,are used to complete and connect the fused edge image.Then the edge image fusion experiment is carried out using Polish Net-2d.The results show that this method can still achieve very good detection results in the case of low texture on the workpiece surface and blurred motion of the collected images.First,we use Polish Net-2d to detect the workpiece in two-dimensional images.In the detected areas,the three-dimensional point cloud in the scene is segmented.The segmented point cloud data are used as the input of Polish Net-3d.The recognition result of three-dimensional point clouds in the area provided by the detection results of Polish Net-2d are compared with the recognition results of Point Net and Point Net++.The results show that the Polish Net-3d proposed in this paper can recognize the segmented workpiece point cloud data based on the detection results of the two-dimensional workpiece detection network,Polish Net-2d,which greatly reduces the impact of the background point cloud data on the recognition rate.Therefore,even when there are multiple workpieces in the scene,the recognition rate of Polish Net-3d can reach 94.50%.