Content-based Tag Processing And Analysis For Internet Social Images

Online social media services such as Flickr and Zooomr allow users to share theirimages with the others for social interaction. An important feature of these services is thatthe users manually annotate their images with the freely-chosen tags, which can be usedas indexing keywords for image search and other applications. However, since the tagsare generally provided by grassroots Internet users, there is still a gap between these tagsand the actual content of the images. This defciency has signifcantly limited tag-basedapplications while, on the other hand, poses a new challenge to the multimedia researchcommunity.As a pioneering work, this paper carries out a series of research efforts for processingthese unqualifed tags, especially in making use of content analysis techniques to improvethe descriptive power of the tags with respect to the image contents.We fully explore the information sources of Internet image data, crowd knowledge ofusers as well as the lexical knowledge base, relying on machine learning, computer visionand information retrieval techniques, to investigate the social images tags and tag-basedapplications. The main contribution and novelty of this paper can be list as follows:We proposed an automatic tag ranking scheme that aims to differentiate the tagsassociated with the images with various degrees of relevance levels. The tags with d-ifferent relevance levels beneft the visual search and in turn improve the relevance oftag-based applications.We presented an image retagging scheme that comprehensively explores the in-terplay of user, data and feature. We further propose a collaborative image retaggingscheme, which propagates each tag over the specifc image similarity graph and couplesthe propagation of different tags through a tag similarity graph. This not only improvesthe retagging performance, but also poses a novel multi-graph multi-label learning algo-rithm to the graph-based learning paradigm.We propose a multi-edge graph to model the multiple relationships among thesemantic regions of two images. By propagating tag information over the graph structure,we naturally achieves the tag-to-region assignment, leading to more fne tag informationwhile improving the reliability of content-based image retrieval. The multi-edge graph was further extended into a more elegant model that supports heterogeneous informationsources as input, and a core equation was developed for implementing the multi-labelpropagation, which fnally enriches the graph-based learning techniques.We propose an interactive image tagging system that achieves a good trade-offbetween the user’s manual efforts and the machine-generated automatic tagging accuracy.This process repeats until satisfactory tagging results are achieved, and users can also stopthe process at any time.