Research On Cross-modal Hashing Retrieval Based On Deep Feature Learning

Due to the rapid development of deep learning and the high efficiency and low cost of hash algorithms,deep cross-modal hashing retrieval has made great progress in recent years.However,the existing cross-modal hashing retrieval methods face the following problems:(1)Most of the existing methods only focus on the feature distribution among the modalities,but ignore the fine-grained information in each modality.(2)The existing unsupervised cross-modal hashing methods seldom consider the unique semantic information of each modality,and it is difficult to effectively obtain the semantic correlation of different modality instances.In order to solve the above problems,this thesis combines attention mechanism and generative adversarial learning on the basis of deep learning,and utilizes graph attention network to bridge the heterogeneity between modalities.The work of this thesis is as follows:(1)We propose a multi-level adversarial attention cross-modal hashing(MAAH).The fine-grained information of each modality can fully express the feature information of the modality.However,most of the current supervised cross-modal hashing retrieval methods only focus on the relationship between the modalities and rarely explore the fine-grained information of the modalities.To solve this problem,We design a modality attention module to find fine-grained information about each modality.It divides modality information into relevant feature representation and irrelevant feature representation using the channel attention mechanism,in which the irrelevant feature representation is the fine-grained information of modalities.Then,we design a multi-level adversarial module to complement the fine-grained information for each modality.In this module,intra-modality adversarial learning can supplement the relevant feature representation of modalities,while inter-modality adversarial learning can make the relevant feature representation distribution of the modalities more uniform.Experimental results on three common datasets show that the proposed method is superior to existing cross-modal hashing methods.(2)We propose a Weighted Graph Attention Hashing(WGAH).Current unsupervised cross-modal hashing retrieval methods seldom explore the semantic relation of different modality instances.In order to obtain semantic correlation of different modality instances effectively,this thesis uses graph attention networks to fully consider the neighborhood structure relation of modalities.The attention mechanism is used to learn different weights of different modality data pairs to realize weighted aggregation of different modality data pairs so as to better mine and improve the relationship between different modality data pairs.At the same time,WGAH also designs a weighted module,so that semantically similar instance pairs have similar distance,and dissimilar instances farther apart,so that the semantic correlation between modality instances can be fully obtained.Experimental results on three datasets show that the proposed method is superior to various deep cross-modal hashing methods.