Research On Image Segmentation Method Of Stacked Electronic Components Fused With Visual Reasoning

In recent years,with the vigorous development of robotics,manual operations have been gradually replaced by machines.In the meantime,the goal of industrial manufacture has changed from automation to intelligence.Improving the autonomy of robots has become the key of intelligent manufacturing.The core of improving the autonomy of robots is to enable them to have the ability of autonomous reasoning.The development of deep learning theory has made artificial neural networks perform prominently in object detection tasks.Further understanding of image semantics based on detection is an important way to realize autonomous reasoning by robots.Image segmentation and visual reasoning,as an intermediate link to understand image semantics,build a bridge for robots to have autonomous reasoning capabilities from scratch.In the automatic assembly process of stacking electronic components,detecting and segmenting each electronic component and inferring the relationship between them is the key to achieving robot autonomy and intelligence.Therefore,this study is based on the convolutional neural network to segment the image of stacked electronic components,through the optimization of the algorithm,realize the instance segmentation of stacked electronic components,and on this basis,explore the application of visual reasoning methods in stacked electronic components.The main research contents and related conclusions of this dissertation are as follows:(1)Research on the method of segmentation of stacked electronic components.Firstly,collect electronic component images and use image processing technology to expand the collected samples,then annotate the images to build a stacked electronic component data set.Secondly,by analyzing the related models of object detection and image segmentation,an instance segmentation algorithm combining depthwise separable convolution and multi-scale feature pyramid network is proposed.Finally,experiments are carried out on the stacked electronic component dataset,and the algorithm in this study,Mask R-CNN algorithm and Cascade Mask R-CNN algorithm are compared and analyzed.Experimental results show that the instance segmentation algorithm proposed in this dissertation has a lighter model,faster detection speed,and better detection and segmentation effects.(2)Research on visual reasoning methods of stacked electronic components.Constructing a semantic feature module and a spatial location feature module,based on the object semantic features and spatial location features extracted by using the feature extraction network,and building a relationship feature module by combining semantic features and spatial location features.Then,the object semantic features,spatial location features and relationships features are combined to build a multi-feature modular fusion mechanism.In this way,the visual relationship between stacked electronic components can be detected.Finally,experiments are carried out to verify the effectiveness of the multi-feature modular fusion mechanism in the visual relationship detection of stacked electronic components.(3)The design of image segmentation system for stacked electronic components.Firstly,the overall scheme of the system is designed,and then the functions required by the software system are clarified.Secondly,the interface of the stacked electronic component image segmentation system is designed,then the software system and the stacked electronic component images are tested to verify the effectiveness of the algorithm that proposed in this study.In summary,this dissertation studies the image segmentation method of stacked electronic components fused with visual reasoning.The image segmentation method and visual reasoning method of stacked electronic components are discussed in depth.An instance segmentation method based on depthwise separable convolution and feature pyramid network is proposed,and the visual relationship of stacked electronic components is inferred based on multi-feature modular fusion mechanism.This dissertation designs and develops an image segmentation system for stacked electronic components,which can realize the function of instance segmentation and visual relationship detection simultaneously.It has important reference value for further improving the robot's autonomy and image understanding ability.