Research On The Relevance Of Multi-modal Data In Cross-media Retrieval

With the rapid development and wide application of Internet technology,the speed of information resources is growing faster and faster.The presentation of media data is becoming more and more diverse,which extends from the original type of texts to different types of texts,images,audio,videos,3-D modal,etc.Different modal media data describe the same event from different angles,which is more expressive.People expect to achieve mutual retrieval between different modal media data.That is to say,by submitting a modal media data,the similar media data from different modality based on the same semantics can be retrieved.Due to the different characteristics of data,different modal media data are heterogeneous on the original over-level features,which leads to not directly to retrieve each other.How to conduct in-depth analysis on the relationship between different modal media data,establish the association between different model media data through the semantics of media data,and measure the similarity between different modal medial data,which has become the focus of cross-media retrieval.Based on the low-level characteristics of media data,this paper fully analyzes the relationship between media data,and proposes two different cross-media retrieval algorithms.Experiments are carried out on several benchmark datasets,and experimental results show the effectiveness of the proposed method.The main work and contribution of this paper are summarized as follows:1.A cross-media retrieval algorithm via joint graph regularization and modal analysis is proposed.This method integrates the one-to-one correspondence between different modal media data pairs,the similarities of the same modal media data and different modal media data into a unified framework.Then different modal media data are mapped into an isomorphic subspace.In this process,different retrieval tasks are treated differently,only considering the semantics of query object can make the learning mapping matrix more pertinent.2.A cross-media retrieval algorithm via discriminant neighborhood and class information is proposed.The method constructs more similar data pairs and dissimilar data pairs by using the category information between data.Based on the semantics of data and the distance between data,this method distinguishes the intra-class neighborhoods and the inter-class neighborhoods.Meanwhile,in order to better use global information of data,the method combines the maximum divergence principle and adds a discriminant analysis preserving term based on the neighborhoods and the semantics into the objective function.Different modal media data can be mapped into semantic space by semantic regression.Finally,the similarity measure is carried out,which returns the closest result from different modality based on the same semantics.