| Due to the explosive growth of multimedia data on the Internet,there is an increasing demand for people to be able to retrieve multi-modal data quickly and easily.Cross-modal retrieval has become a current research hotspot.In order to meet the requirements of low storage cost and high query speed in practical applications,hashing has attracted much attention in the field of cross-modal retrieval.It maps high-dimensional multi-modal data to a common Hamming space for cross-modal retrieval.This article mainly studies how to use deep neural networks to project multi-modal data to a common Hamming space,while reducing coding errors,mining the semantic information of muti-modal data and narrowing the difference between cross-modal data so that the performance of using image retrieves text or using text retrieves image better.In order to achieve better cross-modal retrieval results,this paper proposes the following three hashing techniques for cross-modal retrieval based on deep learning:1.In order to reduce coding errors,Deep Ranking-based Hashing for Cross-Modal Retrieval(DRH)is proposed.This method uses deep neural networks to integrate feature learning and hash code learning into an end-to-end framework,and uses pair-wise loss to reduce the semantic gap between different modalities.In terms of hash functions,this method uses a ranking-based hash function to reduce encoding errors,while using the Adaboost to reduce redundancy between hash codes.2.In order to preserve richer semantic information of cross-modal data,Deep Triplet-based Hashing for Cross-Modal Retrieval(DTH)is proposed.This method utilizes the triplet labels,which describes the relative relationships among three instances as supervision in order to capture more general semantic correlations between cross-modal instance,and uses regularity to solve the problem of hash code discontinuity and maximize the information contained in the hash code.3.In order to narrow the heterogeneity gap between cross-modal data,Generative Adversarial Networks for Cross-Modal Hashing(GANCMH)is proposed.This method models the joint distribution over the data of different modalities by generative adversarial networks.The intermodality and intra-modality correlation can be explored simultaneously in generative and discriminative models.Both of them beat each other to promote cross-modal correlation learning.In this paper,the cross-modal retrieval experiments are carried out on MIRFlickr-25 K,Wiki and NUS-WIDE-10 K.The results show that the three algorithms mentioned in this paper have certain advantages,validity and feasibility compared with the other five state-of-the-art algorithms under the same experimental settings. |
