Locally Affineed Joint-saliency Tree For Co-superpixels Segmentation

Co-segmentation denotes the technique of segmenting several images that contain the same subject and the common foreground simultaneously. This technique can combine the information of all the images and solve the problem in single image segmentation: segmentation without supervision would get less satisfied result due to lack of global information in the images and between the images; segmentation with supervision would need trivial training of the algorithm and make the algorithm very slow. Co-segmentation can combine the information from several images, each image can provide information for the others and compensate the lack of information due to lack of supervision. This technique can be done without supervision and has a moderate speed. But current co-segmentation algorithms mainly focus on segmenting the foreground from the background and paid no attention to the inner structure of the foreground.Recently, co-superpixel segmentation is proposed to focus on the segmentation of the superpixels from each image and find the correspondence of the superpixels, which is a set of pixels with similar texture, location or intensity characters, and always shown as a small region in the image. Superpixel research has attracted lots of attention due to its good performance in reducing the redundancy and maintaining the semantic information. The outputs of co-superpixel segmentation are the superpixels containing the structures in the image and the correspondence with the other superpixels in the other images.Currently, few research methods in the co-superpixel segmentation are proposed but their performances are not so good. Therefore, in this paper we propose a novel algorithm called locally affined joint saliency tree based co-superpixel segmentation algorithm. Our co-superpixel segmentation of image pair can produce superpixels with better correspondence of similar regions between the image pairs. The main contributions of our algorithm are twofold: firstly, we constructed the segmenting tree of the images to fulfill the hierarchical constraint of the image regions, and proposed a joint saliency tree based on the joint salient regions in the image pair, whereby the joint saliency tree can serve as the searching skeleton of the whole joint salient regions to help finding the correspondence of all the superpixels in the image pair; secondly, we performed polyaffine transformations for the superpixels which can help updating the joint saliency tree in each level. In the end, we tested our algorithm with 4 sets of medical images and 2 sets of natural images by comparing it with other co-segmentation algorithms. We also made quantitative evaluation of the experimental results with both matching ratio and matching distance measures. This experimental evaluation showed that our algorithm performs better than the other two methods.