Study On Image Semantic Annotation Via Incorporating Correlations Between Tags

As high resolution digital media capturing devices and mass storage systems become ubiquitous and the network transmission bandwidth increases significantly,the amount of image archives stored in personal collections or shared online is increasing at an exponential speed.How to manage such huge collections of images to enable users to find their interested photos quickly and efficiently is a very challenging task. The traditional content-based image retrieval(CBIR) may not provide sufficiently accurate semantic retrieval performance partly due to the semantic gap.Also,ordinary users often find difficulties in expressing their retrieval intentions in the paradigm of low-level visual features.One way to enable effective sematic image retrieval and provide user-friendly text retrieval interfaces is by building indexes for the images through user labeling or tagging.Unfortunately,the tagging process is often tedious and labor-intensive which is prohibited from being practical.What is desired is a method which can automatically or semi-automatically annotate the images.Therefore,image semantic annotation has great important meaning in both theory and practice,and attracts great attentions from both academic and industrial fields.Traditional pure content-based image annotation models only rely on the visual contents of images but ignore the correlations between semantic concepts to simplify the models.The annotation performance of these methods is often not satisfactory and needs to be further enhanced.With the rise of Web2.0,many content sharing platforms provide annotation functions to enable users to tag images at anytime from anywhere. This increasing availability of user contributed images with tags online provides new opportunities to develop automatic tools to assist the tagging task.In this thesis,we focus on exploiting the correlations between semantic concepts based on the characteristic of image semantic annotation and online tagging.Several related annotation methods are proposed which can be summarized as follows:●A progressive tag prediction framework is proposed to exploit concept contexts. In this framework,the image visual content and the concept contexts are jointly exploited to support image tagging where tags are incrementally added to the image one by one using the image visual contents in the context of previously predicted tags.●A collaborative tag recommendation(CTR) scheme is proposed.Collaborative filtering technique is utilized to tag suggestion based on the tags which have already been provided to the images.Such approach can also be regarded as content-free because it does not directly look into the image contents but rather purely operates on user provided tags.Collaborative filtering is adopted to find the correlation patterns of the semantic labels and extra tags are suggested based on the correlation patterns.●A hybrid probabilistic model(HPM) for unified tag-aided and content-based image tagging is proposed.HPM formulates annotation as a tag suggestion problem under a unified probabilistic framework,where the visual content of an image is represented using a version of bag of visual words representation scheme and the user provided tags are leveraged by exploiting the tag correlations.For images without any tags,HPM predicts new tags based solely on the low-level image features.For images with a few user provided tags,HPM jointly exploits both the image features and the tags to recommend additional tags to label the images.As well as presenting the framework,a collaborative filtering approach is adopted for estimating the tag-to-tag correlations.