| Automatic image annotation is to annotate an image with a set of textual words. The key in this process is to learn a statistical model which correlates the image features with the annotation words. To construct the statistical model, we start with a set of training images, each of which has a set of accompanying annotation words. Typically, images are first segmented into multiple homogeneous regions. Image features such as color and texture are then extracted to represent each image region. The image regions in the whole collection can also be grouped into clusters and thus each image region could be converted into its corresponding cluster id, called a blob. In this way, we obtain a discrete representation of the images. The correlation between annotation words and image features, either discrete or continuous, is constructed with a statistical model. Finally, given a new test image, the same set of image features are extracted, and words are predicated according to the relationship between image features and annotation words established by the learned statistical model.; In this thesis, we explore the automatic image annotation task through a series of statistical models. One model based on the discrete feature representation is the translation model, which constructs the correspondence between blobs and annotation words through a set of translation probabilities. Due to the fact that common words co-occur with many more blobs than rare words, the original translation model overestimates the common words and degrades the overall performance. We thus propose two enhanced translation models to improve the original translation model by incorporating different prior information of the desired translation probabilities into the model. One prior ensures that each word is associated with similar number of blobs, which is measured by the average of the translation probabilities from different blobs to the word. Another prior considers the translation model from two directions: forward translation model, which translates from blobs to words; backward translation model, which translates from words to blobs. The prior specifies that the translation probabilities from these two kinds of models should be consistent with each other. Our empirical results demonstrate the improved performance of the two enhanced translation models over the original translation model.; However, there are still two problems with the translation models. First, they do not consider the correlation between annotation words when making the prediction. Secondly, they are based on the discrete representation, which potentially loses information encoded in the continuous features. However, the correlation information is difficult to explore since the possible number of correlated words is exponential. We propose a Correlated Label Propagation (CLP) framework to explore the correlation between annotation labels. Based on the property of the submodular function, this framework could be solved by a very efficient greedy algorithm and thus be applicable to a large set of labels. In addition, the continuous image features could be incorporated into the CLP framework. Our results show that the CLP framework outperforms the translation models and also can boost the performance to a higher level after the continuous features are incorporated.; In summary, this dissertation shows that (1) The performance of the original translation model can be improved by incorporating different priors; (2) Effectively exploring the correlation information between labels can improve the overall performance; (3) Similarity measurement is very important in label propagation and similarity measurement based on the continuous image features can achieve better performance. |
