Research On Web Image Retrieval Based On The Fusion Of Textual And Visual Information

The rapid development of digital image technology and scan technology and Internet greatly enriches accessible web image resources. Due to the diversity, complexity and irregularity of web resources, how to quickly and accurately find images of interest to users from the huge volumes of the web resources is a very challenging long-term task. To make full use of these web resources, it is necessary to do more research on web image retrieval, including how to organize them reasonably, and how to query and retrieval them effectively. Currently prevalent approaches to web image retrieval fall into two main categories: text-based image retrieval (TBIR) and content-based image retrieval (CBIR).Web images mainly contain two types of information, one is lots of visual information in image contents, and the other is lots of textual information in web pages. TBIR makes use of textual features extracted only from image textual information to index and search images, while CBIR makes use of low-level visual features extracted only from image visual information. To satisfy common user information needing, it is necessary to make full use of the above two types of information in web image retrieval. Furthermore, high-level semantic features should be extracted from image contents, because the degree of satisfaction of retrieved image is judged mainly based on image high-level semantic features. Unfortunately, the extraction of image high-level semantic features and fusion of image textual features and visual features are still a difficult task in the domain of image retrieval.To address the above issues, term similarity measure, a shallow semantic processing technology, is firstly proposed. The research of term similarity measure is one of fundamental research in the domain of natural language processing, focusing on how to quantize term semantic similarity. In this study, term similarity measurement, as the metric form of semantic information, make it possible to compute and compare abstract semantic information existing in human thinking. Furthermore, it is the precondition of the fusion of textual web image information and visual web image information presented by image high-level semantic features.Secondly, a automatic weighted-annotation model for web image is proposed to address the issue of "semantic gap" existing in image low-level visual features and image high-level semantic features: Firstly, it struggles for learning the mapping from image low-level visual feature to high-level semantic feature by means of machine learning and statistical technology; Secondly, the learned map is used to extract high-level semantic feature from image contents; Finally, the quality of extracted high-level features is measured as term similarity based on web textual information and high-level features, resulting the weighted image high-level semantic features which in turn changes the fusion of visual and textual information into the fusion of textual information and image high-level semantic information.After then, we propose a scalable model for web image retrieval. To make full use of textual features extracted from web pages and high-level semantic features extracted from image contents, in this study, the proposed image retrieval model is based on Bayesian inference network. The image textual features and high-level semantic features can be integrated into web image retrieval seamlessly with the help of inference network which has an inherent fusion capability of multiple information sources.Based on the above work, a web image retrieval prototype system is designed and implemented. This system makes full use of two types of web image information as follows. Firstly, extracting high-level semantic features from image contents, then in image retrieval they are integrated with textual features extracted from web textual information. The research results demonstrate the usefulness of the proposed model in web image retrieval.Finally, conclusions and future work are presented.