Matrix Factorization And Semantic Association Hashing For Cross-Modal Retrieval

With the continuous development of information technology,a large number of multimedia data appear in the network,e.g.video,pictures,text.With so much data,people's demand for retrieval has also changed,and people are no longer satisfied with the single-modal retrieval,for example,text search text.Now people expect to retrieve between different modalities,i.e.cross-modal retrieval.In the era of big data,the main challenge in cross-modal retrieval is how to associate different modal data and improve the retrieval efficiency.After years of exploration,research experts proposed a cross-modal retrieval method based hashing,which represent the original data as binary hashing code,and use the characteristics of binary code,such as fast operation speed and small memory space,to achieve cross modal-retrieval of large-scale data.The major contributions of this thesis can be summarized as follows:This paper proposes a cross-modal hashing method based on matrix factorization and semantic association.The advantages of this method are as follows: First,intuitively,there are great differences in the expression of information between different modalities.Most of existing cross-modal hashing methods learn a common latent semantic representation to represent the multi-modal data,whic ignore the the heterogeneity in different modalities,it may let to information loss and unsatisfactory retrieval results.Different with the existing cross-modal hashing methods,we learn modality-specific semantic representation for each modality,then generate corresponding modality-specific hash codes to ensure that the inter-modal relationship of each modal data is not changed.Manwhile,to ensure the consistency between the different modal hash codes,we reconstruct the hash codes by an affinity matrix constructed by semantic labels.Secondly,for the discrete constraints of hash codes,most of existing cross-modal hashing methods use the relaxation quantization strategy to learn hashing codes,which may lead to a lot of quantization errors.In order to solve this problem,we propose a strategy that don't need to relax the discrete constraints in the hash codes learning.In the iterative process,the hash codes is updated bit by bit,which can effectively reduce the quantization error.Thirdly,for the unseen instance,most of existing cross-modal hashing methods generate the corresponding hash code through SVD method,which will lead to numerical instability.To solve this problem,we propose to use kernel logic regression to learn the nonlinear mapping function that can be used for the out-of-samples data.Extensive experimental results on three public benchmark datasets show that the proposed method outperforms several state-of-art cross-modal hashing methods in terms of accuracy and efficiency.