Study On Image Annotation Based On Web Training Data

With the rapid development and popularity of digital devices and Internet,Web image resources have become more and more prosperous.The variety,complexity and irregularity of Web data make it a very challenge task for users to search images they need from such large web resources.One important strategy to address this problem is automatic image annotation,which builds the relationship between image visual content and high level semantics,and then images could be indexed by these annotations.In recent years,the promising development and prosperity of image sharing sites,such as Flickr,makes image annotation an important and valuable research direction.In addition,image annotation is widely used in many applications such as personal album management,medical image retrieval,trademark image retrieval and face recognition.Providing annotations manually requires too much human resources and money, which makes it unrealistic for such large number of images.Generally speaking, image annotation evolves through two stages:the first stage is image annotation based on limited training data,which build the relationship between low level features and high level semantics by applying existing machine learning and object recognition techniques,such as classifiers based methods,cross media based methods,translation based methods and latent topic based generative model;the second one is image annotation based on training dataset from web,which makes use of the abundant resources on the web and extends the scale of training data greatly.This latter strategy is more applicable under the environment of web,and attracts a lot of attention in recent years.This dissertation focuses on several key problems in image annotation based on training data from web.The main contributions of the dissertation are as follows:We discuss the necessity of building annotation dictionary based on web resources.The requirements of annotation words in the dictionary are analyzed for choosing proper words from large scale of words on the web.A random walk model is proposed to build the importance of words in the dictionary,based on the statistical properties on the image sharing websites.In addition,with the use of abundant semantic services on these sharing websites,the disambiguation of initial keywords is studied.By matching the query image with the proper semantic classification,the problem of semantic gap is reduced,which makes the final annotations more coherent.A novel image annotation framework is proposed based on multimodal similarity reinforcement theory.The initial annotations for the query image are firstly obtained by using basic annotation algorithm such as CMRM,CRM.Then the visual content correlations and the annotations correlations are mutually reinforced for computing the final annotations,based on the random walk annotation refinement framework. The final annotations could be more coherent and related to the content of images by incorporating both visual content correlations and annotation correlations.Since the webpages provide abundant semantic interpration for the images on these webpages, we propose to fuse correlations between webpage documents and named entities in the documents,which helps represent the document similarity better.A joint image annotation framework is proposed for annotating personal albums. Different with annotation for a single image,the correlations in the personal albums are considered.The personal album is firstly clustered and the initial annotations are learned from web images for these clusters.The initial annotations are then refined in a semi-supervised learning framework,which combines visual content correlations, annotation correlations and temporal correlations.A cross media based personalized image annotation recommendation model P-DCMRM is proposed.The model combines visual content space,annotation space and user space together.P-DCMRM overcomes the problem of existing image annotation methods,which neglects the visual content of images or the properties of user interests.The global statistical properties and the local ones are both considered for estimating the model.For an uploaded image,annotations could be produced according to different user interests and their annotation history.