Research On Semantic Segmentation Method Of Depth Image Of Mechanical Assembly Based On Deep Learning

The current manufacturing industry is experiencing an era of mass customization,and the changing product types cause frequent reorganization of assembly lines.In this complicated and changeable assembly process,workers are prone to errors in assembly sequence,missing assembly,and wrong assembly.Once these assembly errors are not detected in time,they will directly affect the quality and assembly efficiency of mechanical products,leading to an increase in production costs.The mechanical assembly has a single texture and lacks color information.Compared with color images,depth images not only have the ability to resist lighting,chroma,shadows and other environmental factors,but also reflect three-dimensional information.Therefore,this paper studies the semantic segmentation method of deep image of mechanical assembly based on deep learning.Through semantic segmentation of the assembly depth image,the assembled parts can be identified,and the missing assembly,wrong assembly,assembly sequence error,etc.can be found in time,so as to realize the function of monitoring assembly.In view of the lack of data sets in the field of mechanical assembly,the lack of relevant research on semantic segmentation of mechanical assembly,the poor segmentation performance of small parts,and the blurred edges of segmented images,this paper has carried out the following research:(1)The depth image data set of mechanical assembly is established.First,use the3 D modeling software SolidWorks to establish the 3D model of the mechanical assembly,including the 3D model of each assembly process of the assembly.Then use Mutigen Creator to color-mark each part in each assembly stage of the mechanical assembly model,and then use OSG to establish the depth camera imaging model and the RGB camera imaging model.By changing the viewpoint directions of the depth camera imaging model and the RGB camera imaging model,depth images and label images of different assembly stages and different perspectives are synthesized.(2)A method for semantic segmentation of depth images of mechanical assembly based on multi-skip full convolutional neural network is proposed.This method introduces a skip structure in the second maximum pooling layer and the first maximum pooling layer of the fully convolutional neural network,so that the network integrates more low-level features.Experimental results show that this method achieves excellent semantic segmentation effects on the depth image data set of mechanical assembly.(3)A semantic segmentation method for depth images of mechanical assemblies based on trainable guided filters and multi-scale feature maps is proposed.This method first introduces skip connections in the second maximum pooling layer of the fully convolutional neural network to obtain more low-level features,which makes up for the lack of detailed information in feature map prediction;Then convolution and nonlinear changes are followed immediately after each skip connection operation to deepen the complexity of the network model and improve the data fitting ability of the model;Improved the problem of blurred edges in image segmentation by incorporating trainable guided filters;Finally,multi-scale feature maps are incorporated to obtain parts information of different scales,which strengthens the ability to segment small parts.Experimental results show that this method not only improves the semantic segmentation performance,but also improves the segmentation performance of small parts in the assembly.Compared with other semantic segmentation networks,this method has better segmentation performance on assembly datasets.Semantic segmentation results show that this method improves the problem of blurred edges of segmented images.(4)A lightweight semantic segmentation method for assembly depth images based on U-Net is proposed.This method integrates the improved selective kernel module of this paper on the basis of U-Net,so that the network model can adaptively adjust the size of the receptive field according to the obtained information,and improve the efficiency of the network model to extract features.The improved selective kernel module also greatly reduces the amount of model parameters,reduces the computational complexity,and makes the network model more lightweight.At the end of the network model,the fully connected conditional random field is connected to improve the edge of the segmented image.The experimental results show that,in comparison with other U-Net series of semantic segmentation networks,the method proposed in this paper achieves the highest test intersection ratio on the assembly data set.Semantic segmentation results show that the post-processing process of fully connected conditional random field improves the problem of blurred edges of segmented images.This paper establishes a mechanical assembly depth image data set,and studies three semantic segmentation methods of mechanical assembly depth images based on deep learning.This has certain significance for the realization of automated assembly monitoring and the promotion of intelligent manufacturing.