Research On Image Panoptic Segmentation Based On Spatial Clustering Module And Multi-layer Feature Fusion

As a newly proposed scene understanding task,panoptic segmentation combines semantic segmentation with instance segmentation.Its goal is to assign a semantic category label and an instance identifier to each pixel in the image.Although there have been some works trying to solve the problem of panoptic segmentation,there are still many difficulties.Firstly,most of the current works use heuristic method to combine semantic segmentation and instance segmentation.Although this method is effective,it is slow and complicated to calculate.Secondly,although the panoptic segmentation methods,which solve the semantic segmentation and instance segmentation merging by predicting non-overlapping segments,achieve fast inference results,their performances are still poor compared with the bottomup methods in public benchmark.Therefore,in order to solve the problems of slow speed,complex calculation in heuristic merging method and low performance of the panoptic segmentation method based on nonoverlapping segments,this thesis proposes two image panoptic segmentation methods:double-branch image panoptic segmentation method based on spatial clustering module and panoptic segmentation method based on multi-layer feature fusion.The two methods are described as follows:(1)Considering the complexity of heuristic merging method,this thesis studies the nonoverlapping instance branches,and proposes a double-branch image panoptic segmentation method based on spatial clustering module to reduce the computational cost and storage resources required by the model.Firstly,in order to reduce the network calculations,the lightweight network ERFNet is used as the basic framework of the panoptic segmentation method.Then,in order to generate non-overlapping instance masks and construct instance branch in panoptic segmentation,we use spatial clustering module in the stage of decoder to learn the center of each object and cluster for each pixel,use different loss functions to adjust the object center and the pixel distribution,and train iteratively to achieve regression and classification of image pixels.Next,a simple and fast "majority vote" mechanism is used to merge the semantic branch(the original ERFNet decoder)with the instance branch to generate the panoptic segmentation result.Finally,several groups of comparative experiments are designed on the mainstream dataset of panoptic segmentation to verify the effectiveness and feasibility of this model.(2)Aiming at the loss of image global feature information,which leads to the low performance of panoptic segmentation,this thesis studies the semantic basic framework and proposes a panoptic segmentation method based on multi-layer feature fusion,which further improves the accuracy of panoptic segmentation method by extracting different levels of feature information.First of all,under the condition of the small network parameters,the flexible and lightweight squeeze-excitation module is used into the pyramid pooling,and then an improved pyramid pooling structure is designed.We use this structure to improve the performance of ERFNet,extract the global feature and complete the integration of different levels of feature information.Next,the original basic framework of panoptic segmentation is changed into an improved one to generate panoptic segmentation results.Finally,in order to compare the impact of squeeze-excitation operation at different positions on ERFNet,experiments are performed on the improved basic framework.At the same time,several groups of experiments are designed on the mainstream dataset of panoptic segmentation.The experimental results show that the improved multi-layer feature fusion structure improves the accuracy of the panoptic segmentation network.