Superpixel-Level Joint Saliency Analysis Of Image Group

Through introducing other relative images,we can analyse and use the correspondence between multiple images for improving performance.The main contribution of this paper include:1)We propose a general bag of squares(BoS)model,which provides a full scheme to both invariantly represent superpixels and accurately measure their pairwise similarities.In order to handle the split-and-merge variety of superpixels of same objects in different scenes,our model is based on superpixel pyramid.The BoS model of a superpixel is built upon a group of subregions consisting of the superpixel itself and its children subregions in the pyramid.For each subregion,we extract a proper number of maximum squares via distance transform,and then use a fast self-validated approach to clustering them into a small number of dominant squares,which together with a rotation and scale invariant square descriptor,jointly compose the BoS model for the particular superpixel.2)We propose a co-saliency detection approach based on joint low-rank analysis and online dictionary learning.A unified framework is presented to solve saliency detection in single image and co-saliency detection in multiple images,simultaneously.In this framework,we decompose single image into sparsely salient foreground and low-rank represented background,and co-salient regions with similar appearance are generated by formulating a sparse restoration on a dictionary,which is learned from the salient regions in images of the corresponding image group.3)We focus on producing fast and accurate co-segmentation to image group that is scalable and able to apply multimodal features.We present a general solution for this purpose and specifically propose a non-iterative and fully unsupervised method using point-wise color and regional covariance features for image co-segmentation.The scalability and generality of our method mainly attribute to the superpixel-level irregular graph formulation and multi-feature joint clustering.4)We propose the model to be able to automatically generate visually grounded questions with varying types.Our model is capable of automatically selecting most likely question types and generating corresponding questions based on images and captions.Experiments on public dataset and demonstrates that the proposed method can generate reasonable and grammatically well formed questions with high diversity.