Research On Feature Representation And Ranking For Image Retrieval

Recently,the acquisition of image data becomes more convenient,benifiting from the popularity of mobile devices.With mass storage and diverse ways of propagation,we have witnessed an explosive increase of images.Therefore,it attracts much attention from both the industry and academia how to improve the retrieval accuracy in a large-scale database via an automatic image matching paradigm.In this dissertation,we mainly focus on three aspects in a typical image retrieval system.Our primary contributions are as follows:(1)Aiming to exploit a better usage of pretrained deep learning models,we propose an enhanced algorithm which captures the geometric information in images.Different from previous global feature representations,deep features are extracted on densely sampled image patches,so that the obtained local descriptors preserve the grid in the raw image.Taking the surrounding features of each descriptor as contextual information,we propose a Local Context Aggregation(LCA)algorithm,which aggregates these contextual information to enrich the representative capacity of local descriptors.In order to emphasize features in the key areas,we propose a Centerline Focus Weighting(CFW)algorithm,where local deep features are weighted by their distances to the centerline of the image.Experimental results prove the effectiveness of the proposed LCA algorithm on improving the discriminative power of Convolutional Neural Networks(CNN)features,and the CFW algorithm is demonstrated to be useful for boosting image retrieval performance with weighted deep features.(2)Considering that most previous works take the output of a single CNN layer as a holistic image feature,we propose a Multi-layer Fusion(MF)algorithm to learn a hierarchical representation of different levels.Multiple codebooks are employed for feature quantization separately,and CNN activations are further transformed to deep binary codes by Hamming Embedding.With the proposed MF framework,not only multi-layer deep features can be fused,but also hand-crafted Scale Invariant Feature Transfrom(SIFT)descriptors can be included at the same time.Besides,the differences between images are generally ignored in existing feature fusion algorithms.So we propose a new fusion algorithm,termed as Topo-correlation(Topo),where features are fused in respect to each query.The importance of each feature is measured by cross-view correlations on local affinity graphs,so that the adaptive weights of similarities are node-sensitive,and the fusion procedure is time-saving compared with previous diffusion based graph fusion approaches.Extensive experiments demonstrate the complementary results of multiple features with the two feature fusion schemes,and the proposed algorithms prove to be the new state-of-the-art on benchmark datasets.(3)To solve the problem of invalid image reranking cases caused by insufficient ground truth,we propose a Contextual Modeling on Auxiliary Points(CMAP)method.By isometrically embedding each constructed metric space into the Euclidean space,the image relationships on underlying topological manifolds are locally represented by distance descriptions,so that we can insert Auxiliary Points(AuP)between the query and its nearest neighbor.With these Auxiliary Points,the neighborhood information is densified and enriched.Consequently,reranking based on local relationships achieve more robust outputs.Compared with the pairwise Euclidean distance,the Jaccard similarity coefficient with contextual measure of neighbor sets is more reliable for exploring image relationships on the latent manifold.So we employ the Jaccard index as the reranking measure in the image space with Auxiliary Points.Extensive experiments prove the effectiveness of the proposed reranking method.