Cross-Media Relevance Models Based Image Annotation And Its Application

Automatic image annotation has been an active research topic in recent years due to its potentially fundamental impact on image understandings and that manual image annotation for indexing and then later retrieving image collections is an expensive and labor intensive procedure.In this paper, it aims at untangling this problem in a novel automatic image annotation using cross-media relevance models. Given an uncaptioned image, in terms of a training set of images, keywords, and blobs, it is to develop probabilistic models to estimate the conditional probability between words and blobs by statistic data. Based on CMRM technologies, this framework does not only manage a training stage in a large scale but also catch errors in manual annotation. Comprehensive evaluation conducted on Corel image databases shows the effectiveness and efficiency of the proposed approach.Further, it still remains unclear how to exploit these auto-annotated concepts in image retrieval. This paper tackles this drawback and proposes probabilistic annotation-based cross-media relevance model, and direct retrieval cross-media relevance model, taking advantage of fixed annotation-based retrieval model. The first approach is to annotate each image in datasets using CMRM techniques, index the annotations, and then perform text retrieval in the usual manner. An alternative to fixed -length annotation is to use probabilistic models computing relevant degrees of a given query and datasets to rank images. The last one is to convert the query into the language of blobs and produce the ranked list of images by K-L divergence. Experiments on Corel dataset show that these frameworks can satisfy diverse users.