Attention-aware Joint Location Constraint Hashing For Multi-label Image Retrieval

Large-scale image retrieval tasks are generally abstracted as the approximately nearest neighbor search(ANN).As a representative algorithm of ANN,hashing method is widely used in image retrieval tasks.Recent years,based on the powerful feature extraction capabilities of deep neural networks,a series of deep hashing methods have been proposed,which makes deep hashing methods for multi-label image retrieval a new research field.In this thesis,we focus on multi-label image retrieval based on deep hashing,and strives to solve three problems as follows: precise measurement of the similarity of multi-label images,extraction of representative semantic features of images and consistency preservation between hash codes similarity and original image similarity.We propose an attention-aware joint location constraint hashing for multi-label image retrieval method(ALCH),which conducted the research from three aspects: multi-label image similarity re-definition,image feature extraction and hash network training.The main innovations and results of this thesis are as follows:(1)We propose a novel label joint location constraint similarity(LLCS)definition method to re-define the similarity of multi-label images.By calculating the similarity of objects' location relationship as auxiliary information,LLCS corrects the label similarity to distinguish the similarity of multi-label images more carefully.By leveraging a more accurate image similarity matrix as the supervised information,the retrieval accuracy of hashing codes is improved.(2)We use Goog Le Net as our basic feature extraction model.Based on attention mechanism,we design a two-layer attention sub-network concentrated with basic model to further extract representative feature representations.By weighting the original feature maps in spatial and channel,the output feature maps can mainly focus on the feature subsets that are more relevant to the retrieval task,which helps to generate a more representative image feature representation for the training of hashing network.(3)By correcting the nonlinear range of the original sigmoid function,we propose a new Re-sigmoid function for the training of hashing network,which is more sensitive to the Hamming distance among hash codes.By normalizing the Hamming distance through Re-sigmoid,the similarity of the images in Hamming space and the original space are as consistent as possible.We conducted experiments on four different length of hash codes on three public datasets VOC2007,VOC2012 and NUS-WIDE,and evaluated the retrieval results through four general evaluation metrics.Experiment results show that compared with the eight state-of-the-art methods,our ALCH method achieves a better performance.