Detection-based Instance Segmentation With Shape-aware Features

Modern instance segmentation methods can be divided into two categories,i.e.,detection-based method and segmentation-based method.Leading by Mask R-CNN,detection-based methods always achieve higher precision than segmentation-based methods.Thus,this paper constructs instance segmentation architecture using object detection model.However,the prediction frontend in modern detectors are fully connected layer with RoI Pooling or 3×3 prediction kernels.There exists an issue of misalignment between these two prediction frontends and different scales of objects,i.e.,they cannot capture precise features to detection.Although state-of-the-art detectors with feature pyramid backbone can alleviate this problem,these two prediction frontends still struggle to detect very small objects,and simultaneously,the former continue to encounter the misalignment problem on very large objects.Apart from the defects of the bottom detection architecture,more significantly,there exists heavy occlusion problem between instances of the training images in the instance segmentation dataset.This make instance segmentation been a very challenging vision task.For the misalignment between these two prediction frontends and different scales of objects,this paper proposes attention-guided multi-scale prediction kernels to match different scales of objects.In order to improve detection efficiency,a salient region recognition module is designed to identify these regions that contain confident object cues.There probably emerges many hard negative samples in the detection model,to this end,a novel IoU-adaptive loss function is proposed.Moreover,an interleaved sub sampling feature enhancement method is proposed to fuse fine-grained and highly semantic features and reserve the spatial layout of objects.For the problem of that there exists heavy occlusion problem between instances,this paper proposes shape-aware features for instance segmentation.Concretely,for the occlusion between instances of the different classes,the regional feature affinity is utilized to learn more discriminative feature representation.For the occlusion between instances of same classes,feature direction prediction is employed to learn the location information of pixels in instances,which are with regard to instances' centers.Throughout combining the above methods,the instance model can depict the shapes of instances more precisely.Experimental results show that the proposed one-stage detector can obtain 45.3 AP and the speed of 15.4 FPS on the COCO test-dev with the input image size of 512×512 and using multiscale training strategy,and the AP surpasses all state-of-the-art one-stage detectors.Moreover,it is also comparable with two-stage detectors.Finally,compared with other instance segmentation models,the proposed instance segmentation method more effectively alleviates the occlusion problem and outperforms Mask R-CNN by 2.5 AP^mask.Simultaneously,it also indicates a good generalization ability on Cityscapes dataset.