Saliency Weight And Local Quadrant Constraint For Mobile Visual Search

In recent years, smart mobile devices become more and more popular. People are growing used to obtaining and searching information by mobile devices. Traditional information retrieval methods are based on text keywords. However, they have some problems, such as the limit of describing ability, the influence of different people’s idea, the neglect of important information, and so on. Retrieval based on visual element is a scheme to solve the problems. And it has a lot of potential for the situation that mass information is stored by visual element currently. Therefore the main content of this paper is the researching on the design of mobile visual search system and its performance.This paper design a mobile visual search system based on image saliency and local quadrant constraint. From the functional perspective of view, the system is divided to three big modules: the Android application client module, the PHP intermediate server module, and the background server module. Android application client gets the query image and algorithm flag from users, and then sends them to PHP intermediate server by HTML form. PHP intermediate server receives the content and sends them to background server for applying retrieval service. This paper provides seven algorithms in the background server. While receiving the application of retrieval service, background server selects the algorithm represented by flag to do retrieval for query image. After finishing the retrieval, background server sends the result to PHP intermediate server and then PHP intermediate server sends it to Android application client. Android application client shows the result to users as images list.From the performance perspective of view, this paper proposes a visual search algorithm based on image saliency and local quadrant constraint, which has a competitive overall performance. The Bag of Word(Bo W) model is a popular structure in the large-scale visual search system. Bo W enables large-scale visual search, but it also has some problems. The quantization may reduce the discriminative power of features; and it neglects the spatial relationship in image. To address these problems, this paper introduces the image saliency and image spatial relationship into Bo W. In the Bo W quantization, this paper counts the saliency of features instead of counting number; and in the stage of quantization rectification, this paper uses the sum saliency of visual word to do TF-IDF weighting. In the post-processing stage, this paper introduces the saliency and local quadrant constraint to do resorting. The main idea of local quadrant constraint is that similar objects’ deformations are similar. This paper codes the neighbors of matching features by judging its relative quadrant(Local Quadrant Constraint), and uses the code and saliency of matching features to calculate new similarity score.After developing the system, this paper runs the functional test and performance test respectively. The functional test ensures each function of system is satisfied. The performance test shows the proposed algorithm has an excellent performance.