A Study Of Spatially Regularized Latent Topic Model For Joint Object Discovery And Segmentation In Image Collections

Latent Dirichlet Allocation(LDA) has been applied in the field of image analysis. LDA is established upon the ‘bag of words’ model, which ignores the spatial structure of images. However, the spatial information is usually crucial to some important tasks, such as object discovery, image classification and even segmentation. There exist some works that attempt to address the spatial issue suffered by LDA. One representative model among those efforts is Spatial Latent Topic Model(Spatial-LTM) proposed by Cao et al. Cao incorporates the spatial structure into LDA by assigning neighboring areas with the same object label to preserve the local coherence in images. However, this approach ignores the spatial relations of areas which are spatially distant from each other. In this paper we try to address this issue. To be specific, our contributions can be summarized as follows.1) We present a detailed explanation about the formation of feature dictionaries. We first over-segment each image into multiple super pixels by the SLIC algorithm then extract appearance feature and saliency areas for each segment. We quantize those features into discrete code words by k-means clustering algorithm. We propose a method to find the optimal dictionary size andjustify it through an experiment.2) We propose a hierarchical spatial regularization framework to incorporate the spatial relations of areas which are spatially distant from each other into the Spatial-LTM model. To be specific, we propose a regularization term based on the distance between each pair of super-pixels and append it to the model’s posterior distribution. This regularization framework can effectively preserve the heterogeneity arising from spatial distance in images. We call the improved model Spatially Regularized Latent Topic Model, or SR-LTM.3) We use Gibbs-EM to infer the desirable parameters of SR-LTM and apply it to simultaneous object discovery and segmentation. Also we propose a robust method for automatic selection of the best segment that is most likely to contain the interest objects in an image. We compare the performances of SR-LTM and Spatial-LTM on the MSRC image set and the result shows that SR-LTM outperforms Spatial-LTM in both discovery rate and segmentation accuracy.