Research On Collective Matrix Adaptive Factorization For Cross Modal Retrieval

With the arrival of the of big data era,a large amount of multimodal data has emerged on the Internet.These multimodal data generally include information such as voice,video,text,and pictures.A large amount of multimodal data has bring new application requirements in the field of information retrieval.In the past,using a image query to search for relevant images or using a text query to search for relevant texts.Now,using a image query to search for relevant texts or using a text query to search for relevant images.This new method of information retrieval is gradually attracting people's attention.However,traditional single-modality information retrieval can't solve cross modal retrieval problem.How to effectively achieve mutual retrieval of multi-modal data has become a research focus in the field of information retrieval today.Because hashing technology has advantages of low storage cost and fast retrieval,cross modal hashing retrieval technology has attracted wide attention for multimodal retrieval.Base on the intensive study and analysis of cross modal retrieval,we proposed two cross modal hashing methods to address cross modal retrieval problem.And then,these methods are evaluated on several publicly available data set.The main contributions of thesis are presented as follows:1)Most cross-modal hashing methods only use underlying data structures and latent semantic correlations to measure similarities among data.And to enforce semantic consistency constraints,which is harm to reduce the semantic gap of multimodal data,and it ignores semantic similarity multimodal data.The starting point of semantic similarities preserving in ours methods.Considering the similarities of inter-modal and intra-modal,we proposed relax collective matrix factorization for cross modal retrieval method.The methods proposed that constraint semantic similarity of multimodal data and using matrix factorization build semantic space ofmultimodal data.To reduce cost of computing and semantic gap,this method generate similarities hashing code to realize high-level abstract representation of multimodal data.2)Inspired by deep learning,we can learn higher-level semantics of multimodal feature data through non-linear transformation mapping.Based on this idea,we proposed deep collective matrix factorization methods for cross modal retrieval task.This method builds a multi-level matrix decomposition model to learn high-level semantic similarity expression of multimodal.And then,learning multi-modal similarity semantic hash function by non-linear transformation.The hashing function transform multimodal data into hashing code in semantic space by non-linear transformation and quantitative.