Sparse Representation Based For Cross-modal Retrieval

With the massive increase of various forms of multimedia data(text,images,audio,video,etc.)and the improvement of users' search experience requirements,cross-modal retrieval has become a search trends.In the cross-modal retrieval system,the user can input any modal data,and then can obtain various modal data related to the query data.Finally,make the search result more abundant and satisfy the user's multiple search intent.Considering the storage cost,retrieval efficiency and other issues are the main bottlenecks of large-scale multimedia data retrieval,this paper proposes a multi-graph regularized sparse coding cross-modal retrieval algorithm and variable-length sparse representation cross-modal retrieval algorithm.The coded cross-modal retrieval algorithm was analyzed theoretically and its validity was verified on classic data sets such as WIKI and NUS-WIDE.The innovations in this article include:1.A multi-graph regularized sparse coding cross-modal retrieval algorithm is proposed.Firstly,for the multi-modality unified sparse coding representation(MURL)in cross-modal retrieval,the Laplacian matrix representation data relationship is constructed by using only the tag information,resulting in that the learned sparse coding cannot maintain the spatial topology information of the original data itself.For the problem of weak discriminability,a cross-modal retrieval algorithm based on multi-graph regularized sparse coding is proposed.Then the linear combination of Laplacian matrices of each modal data is added to ensure the sparse coding of the local spatial geometric neighbor relations of the original data.In addition,the linear regression of the label matrix is added to enhance the discriminability of the sparse coding to improve the retrieval.Accuracy is finally matched and cross-modal retrieval within the unified sparse coding space.Compared with the MURL method,on the WIKI and NUS-WIDE datasets,the MAP index values were increased by 22% and 12%,respectively,and were also higher than the typical correlation analysis(CCA)and semi-coupled dictionary learning(SCDL)methods.2.A cross-modal retrieval based on variable-length hash coding was proposed.Hash representation is a special form of sparse representation.Cross-view hash(CVH),Semantic Correlation Maximization Hash(SCM),Semantic Hold Hash(SEPH)and other methods proposed in recent years map multimodal data to the common The hash encoding space is solved by the relaxation of the binary constraint of the hash code.The learned multi-modal hash code is a compromise between the two modalities.It cannot effectively represent the best hash code for each modal data.Makes the category less distinguishable.To solve these problems,this paper proposes a method based on semantic retention to map each modal data to its own optimal-length hash-coding space,and then solves each modal data by the binary constraint-based discrete cross-modality hash algorithm.Optimal hash coding to obtain variable-length hash coding of different modalities.Finally,cross-modal retrieval is achieved by keeping semantically related linear transformations.The proposed variable-length hash-coding cross-modal retrieval model has good adaptability for single-label or multi-label,paired or unpaired multi-modal dataset scenarios.On the WIKI dataset,NUS-WIDE dataset and MIRFlickr dataset,it is verified that the performance of this method is better than the related SCM,SEPH,GSPH and other trans-modal retrieval algorithms.3.A cross-modal retrieval model integrating sparse coding and hash coding is proposed.Although the retrieval based on the hash code method is fast,it may lose the original data information to some extent.In order to maintain fast and efficient retrieval while preserving the original data information as much as possible,this paper combines the advantages of sparse coding and hash coding to do some sparse hash cross-modal retrieval exploration work,and in the WIKI data set.A detailed experiment was performed to show that the retrieval performance has been improved.