Content-based image indexing and query: Feature extraction, evidence combination and relevance feedback

Large visual information systems require effective and efficient means of indexing and accessing images. Content Based Image Retrieval (CBIR) systems have gained prominence over the past few years as a means of indexing and retrieving image data using pictorial examples as query objects. A typical query might entail the location of all images in a database that contain an image similar to a given query image. Most CBIR systems use low level features such as texture, color and shape to formulate the query. A number of methods of extracting features from images have been proposed, almost all of them are domain dependent. There have been a few approaches that combine these features to formulate a more complex query. Relevance feedback (RF) from the environment has also been used as a means of training CBIR systems and improving the performance of feature based query schemes. These methods typically use image classification based approaches, where image query results are returned based on membership of different “classes”. Current systems use probabilistic approaches to adjust the contributions of the different low level features. Query schemes based on feature extraction methods generally make recognition errors of different types, and hence a scheme that would exploit this “error independence” among these schemes could be used to improve the performance of a combined system using these features. Further, if the RF mechanism can be “decoupled” at a system level from the feature extraction methods, new low level feature extraction methods can be added and irrelevant ones removed from the system during training sessions. A new scheme for combining results from a number of low level feature based image classifiers based on the relative relevance of the features is proposed here. This is achieved using evidence combination. A scheme for distributing RF from the environment to each of the low level classifiers is also proposed. This improves the performance of each individual classifier, and in some cases where the feature is not relevant, to remove them from the scheme. The scheme improves the overall relevance of the returned results and hence improves accuracy over other methods.