The Research On Multi-modal Semantic Subspace Mapping Based On Content Features

With the popularity of smart phones and big data,people's demand for multimedia data processing is also increasing.The traditional single-modal processing method can not meet the needs of diversified information comprehensive mining.Therefore,the research methods of multimodal data are gradually being focused by many scholars.One of the most cutting-edge hot spots is the content feature analysis and semantic understanding of multimodal data.Most of the traditional multimedia semantic understanding studies deal with a single type of multimedia data,but it is difficult to mine the latent semantic relationships among multimodal data.Therefore,these methods are less effective in performing multimodal data classification and retrieval applications.How to mine the underlying semantics between heterogeneous data and accurately measure the similarity between features is a key issue in multi-modal semantic understanding.In order to solve the above problems,multi-modal feature subspace mapping based on factor analysis optimization is proposed in this thesis.It obtains mapping matrix and classification matrix by iterative algorithm,and maps image features linearly into text space using mapping matrix.Then it uses the classification matrix for classification in the text space.Considering the nonlinearity of multimodal data and the effectiveness of multinuclear learning,this thesis also applies multi-kernel partial-square regression to multimodal semantic matching,and selects different kernels for different modal characteristics.Multi-kernel partial-square regression analysis is used to map image features to text space,and then semantic analysis of text space is performed using logistic regression.Finally they are applied to the mixed classification and retrieval of images and texts.The experiments on three standard multi-modal data sets verify the validity of the work in many aspects.