Visual Pair-Based Image Search Model And System

With the explosive growth of digital image information,multimedia search becomes important to users. The traditional image search systems always relied on the texts around the images. However, the semantic gap between textual and visual features will decrease the relevance of search results, and content based image search becomes a focus topic in computer vision.There are two drawbacks in the original bag of features model for image search:First, the discriminative power of visual features decreased rapidly when quantized into visual words. Second, the bag of features model considers an image as a bag of visual words, but ignores the spatial information of visual words during the search. To address the first limitation, we present a new approach to generate visual word pairs for image retrieval. SIFT and CS-LBP descriptors are employed to represent the same interest region, and then a visual word pair is obtained by quantizing the descriptor pair with two independent codebooks. By encoding different types of information of the same region, our approach can effectively boost the accuracy of search system. Further, we used two small codebooks in the visual words pair model, which also saved the quantization time. To improve the search accuracy, hamming embedding based on visual word pairs and distance-based soft quantization are introduced into the visual pair model, and the experiment results also verified the effectiveness of these technologies. In this paper, we also implemented a project system of image search, and developed an android application for cover image search. We evaluate our approach with INRIA Holidays dataset on a 120K image database, and the experiment results suggest that our approach significantly improved the retrieval performance of bag of features model,and the runtime of search system is also acceptable for real-time applications.