Theory And Method Research On Semantically Shared Subspace Learning For Cross-Media Data

With the rapid development of information technology, there are many CMD (Cross-Media Data) in the real world. The so-called cross-media data refer to information items with similar underlying contents, which arrive in different modalities, sources or backgrounds, and so on. For example, a webpage describing leopards uses co-occurring text and image of different modalities to represent leopards. These cross-media data show characteristics of the heterogeneity of low-level features and the correlation of high-level semantics. Traditional mono-media learning methods have been unable to adapt to the feature heterogeneity of cross-media data. CML (Cross-Media Learning) is essentially different from mono-media one, which can measure the correlation among different modalities of multi-media objects to implement flexible crossing between different modalities in the learning process. Thus, the research on related problems in cross-media learning must have very important theoretical significance and application value.From the low-level features of cross-media objects, this paper studies some related problems in the cross-media shared subspace learning. It focuses on three aspects below: consistent representations of cross-media data, incremental shared subspace learning, and missing modality completion. In this paper, the innovative research results include:(1) In this paper, a general framework is proposed to discover the Semantically Consistent Patterns (SCP) for cross-media data. Specifically, aiming at building a feature-isomorphic high-dimensional space among different modalities, a novel Isomorphic Relevant Redundant Transformation (IRRT) is first proposed to capture much more complementary information from different modalities. Furthermore, to mine the semantic consistency among the isomorphic representations in the high-dimensional space, this paper proposes a new Correlation-based Joint Feature Learning (CJFL) model to extract a high-level semantic subspace shared across the feature-isomorphic representations. Finally, the proposed framework achieves up to about14%gains in classification and retrieval when compared with other existing methods of the same type.(2) This paper presents an incremental shared subspace learning method for multi-label image classification. With the proposed incremental lossless matrix factorization method, the method can be incrementally performed without using original existing input data, thus high computational complexity involved in extracting the shared subspace can be avoided without decreasing the predictive performance. Compared with other non-incremental shared subspace learning methods, the proposed algorithm reduces computing time about one order of magnitude without decreasing the predictive performance.(3) To complete the missing modality of cross-media data, this paper proposes a general framework. An Isomorphic Linear Correlation Analysis (ILCA) method is first proposed in the framework to linearly map cross-media data to an isomorphic-feature subspace to capture both semantic complementarity and identical distribution among different modalities. Meanwhile, following the idea behind the robust PCA, an Identical Distribution Pursuit Completion (IDPC) model is proposed to accomplish missing modality completion, in which the maximum margin strategy based on identical distribution restraint is fully exploited. Nearly20%gains are achieved for the proposed framework in classification performance than other existing methods of the same type.