Local feature saliency maps for persistent relevance feedback-driven content-based image retrieval

The proliferation of large multimedia databases in recent years presents new opportunities, benefits, and challenges. As databases continue to grow, the increasing difficulty of maintaining such databases is particularly felt in the ability to locate multimedia content in these vast collections. While textual-based searching is a mature research field, linguistic annotations fail to capture the proverbial thousand words represented by an image. Content-based image retrieval (CBIR) is a relatively new research area which seeks to use image content, rather than textual labels, to retrieve items of interest to users' queries.; A new CBIR system is described in this work which uses relevance feedback (RF) to improve its response to image queries from users. A new method of incorporating user feedback to adapt system performance is introduced. This method seeks to more closely align the optimization criteria with the true goals of the system: to parallel human perceptual judgments about image similarity. The method outperforms other known methods of adjusting query responses, and allows similarity-based searches in addition to the typical classification-based searches. The proposed method is particularly well-suited for the case of limited training data commonly encountered in CBIR system design. In the derivation of the new method, a new analysis framework is presented that provides a useful perspective on the capabilities of many RF-based CBIR systems. The proposed system also employs task-based feedback from previous queries to improve new queries—before the user provides any feedback about the new search. This “persistent” learning is possible due to the new paradigm chosen for modeling user intentions. Such persistence is a critical feature for a system to gain acceptance of users in the field. Finally, full response sets from human users are introduced as a mechanism for driving simulated user interaction with the working system and for assessing system performance.