On Hashing Methods For Cross-modal Retrieval Based On Coupled Projections And Dictionary Pair Learning

With the rapid development of multimedia technology,multimedia data represented by text,images,and video has exploded.How to effectively analyze the semantic relevance and similarity measurement of multimedia data has gradually become one of the research hotspots.However,different modal data are represented in different ways,which brings great challenges to cross-modal retrieval.The hash-based cross-modal retrieval method has attracted wide attention due to its fast retrieval speed and small storage space consumption.This thesis studies a cross-modal retrieval method based on the coupling projection and dictionary pair learning methods.The research results obtained are as follows:1.Aiming at the problem that it is difficult to deal with the problem of heterogeneity between different modalities which can't be solved by the traditional hash-based cross-modal retrieval method that directly maps different modal data to Hamming space,we propose a method called Discriminative Hashing Based on Coupled Projections for Cross-Modal Retrieval(DHCP).This method first projects the data of different modalities into their respective subspaces to reduce the "modal gap",then maps the subspace features to the Hamming space to obtain a consistent hash code,and at the same time introduces a linear classifier to strengthen the discriminativeness of hash code,Hamming distance is used to measure the similarity between different modal data.In view of the problem that DHCP does not consider the structure information of the original data,a method named StructuralPreserving Hashing with Coupled Projections for Cross-Modal Retrieval(SPHCP)is proposed,which introduces a graph model in the respective subspace learning to maintain the structural consistency between the data.Aiming at the above two methods that do not consider the semantic relevance between hash codes,a method called Structural and Semantics-Preserving Hashing with Coupled Projections for Cross-Modal Retrieval(SSPHCP)is proposed.In this paper,the retrieval performance of the proposed algorithm is tested on multiple public data sets,and the experimental results show that the proposed algorithm can significantly improve the,mean average precision.2.Aiming at the problem of poor interpretability and time-consuming training of crossmodal retrieval methods based on synthetic dictionary learning,a method called Discriminative Dictionary Pair Learning for Cross-Modal Retrieval(DDPL)is proposed.Firstly,DDPL constructs the dictionary pairs of the respective modal data from original data and learns the sparse representation of different modal data through their respective dictionary pairs,and then map the sparse representation to the Hamming space to obtain a consistent hash code,at the same time class label constraint is introduced to improve the discriminative of hash code Finally,the Hamming distance of the hash code is used to retrieval between cross-modal data.Aiming at the problem that the DDPL algorithm does not consider semantic consistency,a method named Discriminative and Low-Rank Dictionary Pair Learning with Semantics-Preserving for Cross-Modal Retrieval(SPDLRDPL)is proposed.On the one hand,the SP-DLRDPL algorithm constructs a similarity matrix to maintain semantic consistency.On the other hand,in order to reduce the impact of potential noise in the data,low-rank constraint is used when learning the synthetic dictionary.The corresponding experimental results show that the proposed algorithm can effectively improve the retrieval accuracy.3.Aiming at the common "modal gap" problem in multi-modal retrieval,this thesis takes advantage of the idea of DHCP and DDPL algorithm and extends the two algorithm to multimodal retrieval In this thesis,a multi-modal retrieval algorithm based on coupled projections and a multi-modal retrieval algorithm based on dictionary pairs is proposed.The corresponding experimental results show that the above two algorithms can effectively deal with the multi-modal retrieval problem.