Research On Panoptic Segmentation Based On Deep Learning

In recent years,the development of artificial intelligence has been deeply involved in various fields of society,such as intelligent security,autonomous driving and medical technology.Algorithmic research based on deep learning has made outstanding contributions in the field of artificial intelligence development.Computer vision is a hot research topic in the field of artificial intelligence development,and image segmentation is one of the most important topic in the field of computer vision.At present,image segmentation technology mainly includes semantic segmentation,instance segmentation and panoptic segmentation.Semantic segmentation is the class distinction of each pixel in the image,instance segmentation is the detection of the target in the image and pixel-by-pixel segmentation,panoptic segmentation is the detection and segmentation of all objects in the image,including the background.As there is still a lot of optimization space for panoptic segmentation algorithm,aiming at the shortcomings of the current algorithm,this paper mainly conducts research in the following two aspects:At present,most panoptic segmentation networks manually adjust the weight parameters of the loss function when fusing semantic segmentation branches and instance segmentation branches,so this paper introduces a panoptic segmentation method based on multi-task learning.In addition,this paper also introduces spatial sorting module for the current panoptic segmentation algorithm to ignore the overlap between instances,which prioritizes overlapping instances according to classes.The network model first uses the feature pyramid network to extract the backbone features,and inputs the extracted features into the semantic segmentation branch and instance segmentation branch respectively;Then the model introduces the information sharing flow between the two branches to realize the information sharing;Finally it uses the spatial sorting module to fuse the prediction results of the two branches,and the multi-task loss function is used to optimize the system parameters during the fusion.The model of this paper is trained,verified and tested on the MS COCO dataset and Cityscapes dataset,and good segmentation results are obtained.Aiming at the problem that most of the current panoptic segmentation networks are two-stage segmentation based on detection,an efficient single-stage panoptic segmentation network model based on the attention mechanism is proposed.The network completes panoptic segmentation tasks with fast reasoning time.The framework shares the prototype mask with semantic segmentation branch and instance segmentation branch to generate a new instance mask.In order to improve the quality of the shared prototype mask model,a cross-layer attention fusion module is used to perform multi-scale fusion of the features output from different layers of the feature pyramid network,then input to prototype mask module.Finally,the model of this paper is trained,verified and tested on MS COCO dataset,which can achieve better system performance with faster reasoning speed.