Research On Semantic Analyzing And Retrieving Large-scale Image Collections

With the rapid development of World Wide Web, the prevalent availability of various digital equipments, and ever-growing improvement of large-scale storages, we have witnessed the explosive growth in the amount and complexity of image data. Micro digital images form the basis of many applications, like entertainment, commerce, education and so on, bing many image databases that tend to grow into unwieldy proportions. The management, retrieval and application of large-scale image information has observed more and more attention from researchers, it has become a challenging problem to efficient and accurate search and retrieval form the explosive growth of image data.A variety of Content-Based Image Retrieval(CBIR) techniques have exploited visual features of images, such as colors, textures, and shapes to represent and index image content. CBIR initiated in the 1990's, which can provide automatic and objective representation of image content tools for querying image, becomes more and more popular in research area. However, it is widely noted that there is a "semantic gap" between the visual features and the semantic meanings of images. The ''semantic gap" has been a major problem for most CBIR approaches. In addition, due to the computation complexity of visual feature extraction and the "dimensionality curse" for high-dimensional visual feature retrieval, CBIR has not yet seen a successful commercial application. Semantic-Based Image Retrieval, aims to capture high-level abstraction and concepts of image data, focuses on the derivation and analysis the user's interpretation information, and take advantage of high-level semantic interpretation in finding relevant images, which promises important benefits CBIR. Due to the complexity and subjectivity of image semantic information. it still exists many technique challenges in semantic derivation, semantic representation, and semantic comparison. Image semantic retrieval is still an important and challenging research project in the multimedia information retrieval domain.To tackle the main issues in large-scale image semantic retrieval, in this paper, our researches focus on the following key techniques: correlative analysis of image features, semantic knowledge description, semantic similarity metrics, semantic search paradigm, query interpretation, search results clustering and so on. The proposed techniques are integrated to implement an novel, intact, and highly effective system for large-scale image semantic retrieval. The main contributions are as follows:(1) We give an in-depth study of the interpretation and appearance of ''semantic gap", and extract the semantic information hierarchically. By employing semantic analysis, we propose an hierarchical semantic model for large-scale image databases based on the knowledge and criteria in the field of linguistics, domain and user feedback, which can capture the cause of formation of "semantic gap" in different semantic level, The main advantages of this structure include the following: a) This model can capture a collection of concepts for large image database, and provides a dynamic snapshot for their interrelationships, which enrich the scope and facets of image semantics. b) The complexity and diversity of semantics make it difficult for deriving semantic information directly, the top-down fashion construction of hierarchical semantics provides a more natural way for people's recognition, and captures multi facets and multi grades interpretation of image semantics.(2) The index based on semantic in non-metric space and the index of visual content in metric space are explored: Firstly, we propose an image similarity metrics by integrating the high-level semantic knowledge and the low-level visual features, based on the study of image semantic similarity from different semantic level(visual information, meta-semantic, high-level semantic and image semantic category). And then, we present a novel image query algorithm based on the proposed similarity measure. This main advantages can be summarized as: a) The textual descriptions usually carry more semantics of images compared to the visual features, while the visual comparison is effective only when the semantics of the images being compared are well correlated. The mode of comparing visual features in the second phase is able to accurately characterize and quantify perceptual similarity. b) For large image datasets, this two-phase query technique first uses the textual semantic to locate the relevant image semantic subset, and then combines the visual similarity to search for the final relevant images. By fast reducing the search space, it not only outperforms the conventional system in orders of magnitude, but also achieves high retrieval accuracy.(3) The proposed techniques are implemented in a novel system called HISA, which provides highly scalable organization for very large image databases and support multi form of user query: By integrating a set of the proposed techniques, like image feature extraction, hierarchical image semantic analysis, the construction of image semantic model, semantic similarity measure, the searching paradigm etc., HISA support the queries by either keywords, image examples, or both. Browsing for images in this system can use an automatic maintained category, and employ automatic image annotation technique to enhance the searching ability. We conduct extensive experiments using image datasets containing a variety of real-world images to evaluate our HISA systems in terms of various parameter tuning text, the comparison with traditional algorithm and the case study. The experiment results show that our proposed techniques are effective in achieving high run-time performance with improved retrieval accuracy The experiments also show that the proposed method has good scalability.(4) We study new techniques to effectively retrieval images in tag-space by bringing together semantic meanings implied in tag corpus. We propose a novel iterative searching and query refinement model in tag-space named Pivot Browsing: By integrating query expansion techniques, tag clustering algorithm, visual re-ranking method, and user feedback, this model provide a novel and user friendly interface for image information navigation and user interaction, and facilitate user to browse numerous and complicated returned results, fast locate the wanted images more accurate, and perform knowledge discovery. Especially, we employ a top-down graph partitioning algorithm to achieve fast and effective image results clustering, which can get the on-line performance. We believe that Pivot Browsing can potentially become a general tag-space search paradigm not only limited to images.(5) Base on the above studies, we implement a novel image retrieval prototype system called PivotBrowser in tag-space, and we also evaluate the effectiveness and usability of Pivot Browsing by extensive experiments. PivotBrowser support multiple user operations and feedback manners, and provide novel mechanisms for iterative query refinement and result presentation via an innovative user interface. We perform comprehensive evaluation of our proposal over a large tagged image dataset to give an overall evaluation of Pivot Browsing, including qualitative case study and quantitative performance analysis. Experiment results reveal that PivotBrowser is effective and efficient for tag-space image retrieval in terms of user-rating and runtime performance. The case study shows how PivotBrowser address the semantic limitations in tag-based image retrieval, and the effectiveness of results clustering, the qualitative analysis and user-rating show how PivotBrowser improves query accuracy and user experience in image exploration and retrieval.