Color-spatial image indexing and applications

We propose a new image feature called the color correlogram as a generic color-spatial indexing tool to tackle various problems that arise in content-based image retrieval and video browsing. Informally speaking, a correlogram represents the spatial correlation of colors in an image. While the computing and storage costs of correlograms match those of histograms, the presence of spatial information makes the former more stable to tolerate large image appearance changes than the latter. This makes the correlogram very attractive for applications such as content-based image retrieval and cut detection.; To validate this, we first show that the correlogram, used as an image feature, is scalable for image retrieval on very large image databases. Our experimental results on a database over 200,000 images suggest that the color correlogram is much more effective than the color histogram (and variants) with the same amount of information for these applications. We also propose a new distance metric called relative distance metric for comparing image feature vectors. It outperforms other distance functions in most cases and improves the performance of color histograms and histogram-based features. To further enhance the quality of retrieval, we then present two supervised learning methods--learning the query, learning the metric--and combine these learning methods with color correlograms. Our experiments show that these learning methods are quite effective with even a little effort from users.; We also adapt the correlogram to handle the problems of image subregion querying, object localization and tracking. We propose the correlogram intersection for object detection and correlogram correction for object localization. These simple methods perform better than methods based on color histograms.; Finally, we propose a method for hierarchical classification of images via supervised learning. This scheme uses correlogram as the low-level feature and performs feature-space reconfiguration using singular value decomposition to reduce noise and dimensionality. We use the training data to obtain a hierarchical classification tree that can be used to categorize new images. Our experimental results suggest that this scheme not only performs better than standard nearest-neighbor techniques, but also has both storage and computational advantages.; All our experimental results suggest that the color correlogram can serve as a good generic indexing tool for various image and video processing applications. Thus, it promises to be a basic building block for efficient and effective schemes to retrieve images from say, the world-wide web.