Few-shot Semantic Segmentation With Capsules

Semantic segmentation refers to classifying an image at the pixel level and marks a semantic category for each pixel in the image,which is one of the main research directions in computer vision.It has applications in many fields,such as intelligent self-driving cars,auxiliary medical diagnosis,geographic information systems,etc.With the development of deep convolutional networks,the performance of semantic segmentation has also been greatly improved.However,the CNN-based semantic segmentation methods require a large number of labeled images.In most practical scenarios,we cannot effectively obtain many full-labeled samples.Therefore,the few-shot semantic segmentation was proposed in 2017.Few-shot semantic segmentation aims to effectively perform semantic segmentation under the condition that there are only a few labeled training images,and can be extended to the unseen classes quickly.This task divides the data set into a support set and a query set.The support set and its labeled image and query set are sent to the network at the same time,and the network output the segmentation result of the query image.The current few-shot semantic segmentation models mainly extract high-level features through a pre-trained classification network and then use the cosine similarity map of the high-level features to guide image segmentation.However,the current methods still face problems such as misleading similarity maps and insufficient utilization of high-level semantic information,which leads to inaccurate segmentation results.In response to the above problems,this paper improves the existing few-shot semantic segmentation method based on a similarity map.The main contents of this paper are as follows:(1)We propose a one-shot semantic segmentation method based on capsules.To tackle the problem of misleading cosine similarity maps,we use the pre-trained Res Net50 as the feature extractor to obtain the high-level features.Besides,we transform the features into capsules by squash function.Then,a capsule-based similarity map method is constructed,which effectively improves the performance of one-shot semantic segmentation.The experimental results on PASCAL-5~i and COCO show that the method is effective and reliable.(2)We propose a few-shot semantic segmentation method based on capsules.To solve the problem of insufficient utilization of high-level semantic information,we introduce a margin loss function suitable for capsules.Finally,the experimental results show that our five-shot method can achieve better segmentation results than the one-shot segmentation method.