Research On Image Retrieval Based On The Fusion Of Visual And Semantic Information

The study of image label is one of the most important and critical steps to realize thesemantic-based image indexing, retrieval and other related applications, which aims to establish aprecise correspondence between visual information and semantic description. Tagging is foreseenas a method to bridge the semantic gap in image analysis.Tag-based retrieval, which returns images annotated with a specific query tag, is an importantway of searching or browsing images on Flickr. However, the existing ranking methods fortag-based image search frequently return results that are irrelevant or low-quality. It is argued thatthe relevance and quality are two important measures for a user friendly summarizing the returnedimages. In this paper, we propose a relevance-quality ranking method considering both imagerelevance and image quality. First, a relevance-based ranking scheme is utilized to automaticallyrank images according to their relevance to the query tag, which reckons the relevance scores basedon both the visual similarity of images and the semantic consistency of associated tags. Then,quality scores are added to the candidate ranking list to accomplish the relevance-quality basedranking. Experimental results on NUS-WIDE image collection demonstrate the effectiveness of theproposed approach.It is worth noting that although we have only used Flickr data in this work, the proposedrelevance-quality ranking method for tag-based search result is a general approach and can beapplied for other social media sources (e.g., Youtube and Zooomr) as well.Finally, we conclude the whole thesis and discuss future directions of research that could furtherboost the performance of the image retrieval algorithm. Notice, one strength of our algorithm is thatonly using very simple features for image qualities, and we achieve very good results. It is certainlypossible to improve with more sophisticated design of features. Besides, it is still difficult toeffectively and automatically fuse different model obtained from diverse information channels fordifferent applications. Therefore, a promising future direction is to learn an intermediaterepresentation that maximizes the correlation between the visual content and semantic tags.