Research On Cross-media Retrieval Based On Correlation Analysis Of Sparse Coefficients

With the rapid development of Internet technology,a large amount of multi-modal data(such as images,text,video or audio)is widely used.Prior to cross-media retrieval,single-media retrieval technology is a popular retrieval method.This retrieval method is relatively simple and does not support cross-media retrieval between different media types.As a result,cross-media retrieval is becoming more and more important.In cross-media retrieval,users can submit any type of media object to query media objects of all media types.Multimodal data usually has different feature spaces.The heterogeneity between different modal features is a huge challenge for cross-media retrieval.The most direct way to solve this problem is to map different modalities data to a shared space,the similarity between different modalities in the shared space can be directly measured.However,when learning shared space,most existing methods only consider the direct correlation between different modalities,ignoring the inherent diversity of their representations and related structures;neglecting the selection of correlation features in the mapping process and the relationship inter-modality and intra-modality.Through the coupling method,the dictionary learning is used to obtain the sparse coefficients of different modalities.The sparse coefficients of different modalities are homogeneous,and the relationship between the modalities is combined to ensure a more representative shared space.In the mapping process,a 21 norm terms is applied to the mapping matrix,and graph regularization is used on the data to ensure feature selection and modal relationship.In this paper,two methods of cross-media retrieval are proposed.Experiments are carried out in two data sets.The experimental results of cross-modal retrieval tasks show the effectiveness of the proposed method.The main research work of this paper is as follows:(1)A cross-media retrieval of joint coupled dictionary learning and image regularization is proposed.The method includes: 1)coupling dictionary learning steps.A uniform sparse representation is generated for different modalities by associating and jointly updating dictionaries of different modalities;2)coupling feature mapping steps.Projecting sparse representations of different modalities into a common subspace defined by class label information to perform cross-modal matching.In the coupled feature mapping process,21 norm terms are applied to the projection matrix,and the correlations and discriminative features from different feature spaces are selected;further use of graph regularization terms for data,retaining similar relationships of inter-modality and intra-modality.(2)A cross-media retrieval of joint feature selection and latent subspace regression is proposed.The method firstly projects multimodal data into a common subspace,measures the similarity between different modalities and then retrieves it.Applying 21 norm penalty terms to the projection matrix,can select correlation and discriminant features from the feature space.At the same time,spectral regression method is used to learn the optimal latent space orthogonal constraints of all modal data sharing.Then construct a graph model to project multimodal data into the latent space,preserving the similarity relationship within the modality.