A Research Of Weakly Supervised Learning Of Semantic Segmentation Algorithm

Deep learning and Convolutional Neural Networks(CNNs)have recently achieved great success in computer vision,such as semantic segmentation.Driven by the requirement of scene recognition,various models have been proposed to accurately segment foreground in images.However,manual annotation for training semantic segmentation network demands massive financial investments and is a time-consuming effort.To alleviate the heavy dependence on pixellevel annotations,weakly supervised learning for semantic segmentation is adopted,which uses weak annotations in semantic segmentation,including bounding boxes,scribbles and imagelevel labels,etc.Among all the supervisions above,image-level label is widely used as it is available in most datasets(e.g.,VOC and MS COCO).Even if using image-level label is common and convenient,there exists a critical issue when classifying each pixel only with image-level class labels.The classification task requires translation invariance but the semantic segmentation task is position-sensitive and requires translation variance.To address this issue,CAM(Class Activation Maps)is proposed to overcome the inherent gap between classification and segmentation by adding a global average pooling in the top of fully convolutional network to get class localization maps.However,this architecture tends to activate most discriminative object regions and obtains incomplete segmentation results.In recent works,CAM is usually taken as an initial localization technology followed by additional methods to refine it.However,most approaches focus on propagating foreground regions but do not consider the coincidence of segmentation boundary and real boundary,which is a main limitation for segmentation performance.In many cases,the segmentation might be stretched into irrelevant regions if its propagation is not properly constrained by object boundaries.In this paper,we propose a weakly supervised learning based semantic segmentation approach,that will output corresponding pseudo semantic segmentation label for each image.The main contributions of the paper are summarized as follows:(1)As the image-level label has no location cues to support semantic segmentation prediction,we decide to do class localization firstly.In this paper,we introduce two kinds of localization methods,including model based localization techniques and back propagation based localization techniques.We discuss related works of localization technique and propose Attentionpooling CAM and a novel back propagation principle.(2)We propose an optimization method to refine coarse semantic segmentation result.Concretely,the segmentation result is used to synthesize boundary labels and train a network named EBNet to predict a boundary map for each image,finally,the boundary map is converted into transition matrix to direct the propagation of object segmentation.(3)With the combination of Attention-pooling CAM and refinement method,we get a two-stage method named BES(Boundary Exploration based Segmentation).To demonstrate the efficiency of proposed method,we use generated pseudo semantic segmentation labels to supervise DeepLab v1 and Deeplab v2,segmentation results in PACAL VOC 2012 achieve the state-of-the-art performance.