Semantic Segmentation-based Method For Monitoring The Assembly Process Of Mechanical Assemblies

Assembly is an important process in the manufacture of mechanical products.The quality of assembly directly affects the quality of the product.However,mechanical assembly is various connection forms and tedious assembly processes.So workers are prone to part incorrect and omission assembly in the assembly process.Therefore,this paper investigates the assembly process monitoring method based on semantic segmentation.The method performs semantic segmentation of assembly images,identifies assembly parts,and replaces manual monitoring of the assembly process.The details of the research are as follows.(1)Using the analysis classical fully supervised semantic segmentation methods,Deep Labv3+ network with better segmentation performance on public datasets is selected as a fully supervised semantic segmentation network for assembly images.An assembly RGB real image semantic segmentation dataset is built for training the Deep Labv3+network,and the results of semantic segmentation are visualized.The experimental results show that Deep Labv3+ has a better segmentation effect on the semantic segmentation dataset of assembly RGB real images.(2)An improved semi-supervised semantic segmentation method for assembly images based on adversarial training and adaptive feature fusion.The method incorporates the ASFF multiscale output fusion method to fuse the outputs of different dimensions of ASFF into one output in order to fully utilize the high-level semantic features and the low-level fine-grained features,which helps to improve the segmentation accuracy of the model.The RFASPP module is improved to fuse RFB and ASPP structures to expand the perceptual field so that the network can extract more deep features.A Coord Conv module is added to enable the convolution to sense spatial location information,making the semantic segmentation network with good location sensitivity.In the discriminative network,spectral normalization is introduced to increase the stability of training,which in turn improves the accuracy.Experiments show that this method outperforms semi-supervised semantic segmentation networks such as U2 PL on the assembly depth synthesis image dataset.(3)A semi-supervised semantic segmentation method based on consistency training is investigated to improve the PS-MT network for features such as uneven part sizes of assembly images.In the PS-MT network,another Teacher branch is added to improve the stability of pseudo-labeling.In the PS-MT network,multiple dimensions are added to perturb instead of adding perturbation in one dimension in the original network,which further enhances the consistency learning efficiency.In the PS-MT network,the feature fusion module is added to bridge the spatial and semantic gap between low-level features and high-level features.In the PS-MT network,the contrast regularization loss function is introduced to prevent the model from overfitting the noise labels.Experiments were conducted on the assembly synthetic image dataset.The experimental results show that the above proposed improvements to the PS-MT network are beneficial to improve the segmentation accuracy of the PS-MT network.(4)Using Paddle X and Qt tools,software was developed for monitoring the assembly process of gearboxes to monitor the part assembly process.The software semantically segments the captured video and analyzes the segmentation results in order to monitor the assembly sequence.Taking the reducer as an example,the software loads a Deep Labv3+ model trained on the Assembly RGB Real Image Semantic Segmentation Dataset,and conducts the monitoring experiments on the assembly process of the reducer.The experimental results show that the software can prompt the assembly sequence and give corresponding hints if the assembly sequence is wrong,so as to realize the function of assembly process monitoring.In addition,the software can be loaded with trained network models of different assemblies to quickly build a software system for monitoring the assembly process of different products,which has a certain degree of versatility.This paper researches the assembly process monitoring method based on semantic segmentation,realizes the mechanical assembly image segmentation method based on fully supervised and semi-supervised semantic segmentation,and develops the assembly process monitoring software,which is meaningful to realize the intelligent monitoring of the assembly process and promote the intelligent development of manufacturing industry.