Research On Cross Modal Retrieval Method Based On High-order Semantic

With the development of information technology,the information of data is growing explosively,and its presentation forms are also increasing.The presentation forms of the data are gradually expanded from text presentation to multimodal forms,such as image,voice,video and so on.Multimodal data is interrelated and complementary,which enriches the forms and content of information.The traditional single modal retrieval method between text and text has been unable to adapt to the rich and diverse information data.The cross-modal retrieval between text-image,image-video and other modals is more in line with the current needs of people.It is of great significance to carry out cross-modal data retrieval.The main problem of cross-modal retrieval is that there is a semantic gap between multimodal data,which means data in different modals represent the same semantic information but with different presentations and dimensions.So it is difficult to compare the measurement between different modals directly.Aiming at this problem,we take two modals of text and image as the research object?other modals can be analogized?,and we study how to improve the accuracy of cross modal retrieval from the high-order semantic correlation between modals in this thesis.The main contents and innovations of this thesis are as follows:?1?Based on the semantic annotation information and the sparse structure requirements of multimodal data,a subspace cross modal retrieval method based on high-order semantic correlation is proposed.The semantic annotation information of multimodal data is processed to extract the high-order semantic correlation,and the L₂₁norm is introduced as the sparse regularization operator to meet the needs of joint feature selection.Multimodal data is mapped to a common subspace based on the semantic correlation and data sparsity constraint,so that image and text data with heterogeneous structure are mapped to dimensionless data,which can be compared directly,and the accuracy of cross-modal retrieval effectively can be improved.?2?Based on the cross modal retrieval method of high-order semantic correlation subspace,a subspace cross modal retrieval method based on hypergraph ranking is proposed.This method is mainly divided into two parts:subspace mapping and hypergraph ranking.In the part of subspace mapping,the high-order semantic correlation and sparse structural constraints of multimodal data are considered.In the part of hypergraph ranking,the characteristics of multiple correlations of multiple objects can be described by hypergraph,and the similarity of inter-and intra-modals is considered comprehensively to mine the similarity between multimodal data furtherly.The combination of hypergraph ranking and subspace learning can further mine the semantic correlation between cross modal data,which can further improve the accuracy of cross modal retrieval.