Cross-modal Retrieval Based On Domain Adaptation

With the development of the Internet,a large number of multimedia files are produced every day.Traditional single-mode retrieval can no longer meet people's needs,such as image retrieval and text retrieval.Therefore,cross-modal retrieval has gradually become the focus of scientific researchers.Cross-modal retrieval regards one type of data as the query and search for the similar content files of another modality.Compared with traditional retrieval methods,the results of cross-modal retrieval contain more abundant information,which brings people a good retrieval experience.However,different modalities are inconsistent on distribution and representation,the existed “modality gap” makes it hard to directly measure the similarity of samples.Thus,reducing the heterogeneous differences among various types become the challenging problem to be solved in cross-modal retrieval task.Inspired by the idea of feature migration in domain adaptation,we integrate domain adaptation ideas into common subspace learning and extract domain invariant features,so that we can directly calculate the correlation among different modal samples in subspace and obtain the corresponding retrieval results.In this paper,two algorithms are proposed to solve the cross-modal retrieval problem.The main innovations and contributions can be summarized as follows:1.Traditional subspace algorithms focus on the structural information within the same mode,and ignores the complementary information among various media files.In order to extract the relationship among various type data,we propose a new subspace learning algorithm for cross-modal retrieval task in the third chapter.We employ effective regularization terms in our framework to simultaneously minimize the classification error and enhance the relationship between modalities.The extensive results on three benchmark multi-modal datasets show that our proposed method performs better than some relevant state-of-the-art approaches.In addition,we do deep analysis of the algorithm,and increase the experimental content of convergence speed,time complexity and parameter sensitivity.The experimental results show that our proposed algorithm has the advantages of low time complexity,stable effect and fast convergence.2.We propose a multi-space learning algorithm for cross-modal retrieval task in the fourth chapter.Unlike traditional subspace learning,our proposed method elegantly combines common subspace and private latent learning into an end-to-end network,which store the structural information within the modality and reduce the noise of the shared subspace.In addition,in order to make private and public space independent of each other,we add orthogonal constraints on private and public space,and use reconstruction error to ensure the effectiveness of multi-space learning.Compared with other algorithms,our proposed algorithm has achieved good performance in both retrieval tasks.In order to prove the effectiveness of each part in the algorithm,we test different combinations of multiple parts.The experimental results show that the coordinated use of multiple spaces is conducive to the improvement of retrieval effect.In addition,we also add some experimental contents such as parameter sensitivity analysis and convergence rate analysis in the experimental part.