The Cross Media Retrieval Based On Correlation Analysis

In order to accord with the human brain through a comprehensive treatment mode of different senses to outside information cognition and perception,so that the computer can simulate the human brain to process multimedia data of different types of cognition,learning and decision-making information processing,so the“cross media”technology came into being.With the explosive growth of multimedia data on the Internet and more diversification and complexity,it is increasingly important to achieve accurate and effective mutual cross-media retrieval.Cross-media retrieval is currently faced with two major challenges: On the one hand,due to the different types of multimedia data in its underlying features that there are differences in the characteristics of the dimension and attributes,resulting in heterogeneous problems between each other,which is crossmedia heterogeneous gap;On the other hand,cross-media data is often associated with the semantics of its expression,but there may be inconsistencies between the underlying characteristics of different types of multimedia data and its high-level semantics,which is cross-media semantic gap.In order to solve these two problems,we need to dig and analyze the complex relationship between the cross-media data.In this thesis,the structure of the deep canonical correlation analysis is improved and applied to cross-media correlation learning model.Deep Canonical Correlation Analysis(DCCA)is a deep method that maps text and image pairs to public latent subspaces for similarity measurements.The proposed method improves the structure of the traditional DCCA,the first layer of hidden layer in the network is transformed into a linear projection loss layer.The training of the linear projection layer is combined with the training of the nonlinear hidden layer to ensure that the linear projection can be well matched with the non-linear processing stage,at the same time,the original input data can be obtained from the output of the network more abstract and more accurate representation.And then through cross-media correlation analysis to dig out the complex correlation between the cross-media data.In the latent space of cross-media retrieval,semantically consistent images and texts should be close to each other,but use the CCA and its extension method to maximize the correlation between the image and the corresponding text does not meet this requirement.Therefore,this thesis proposes a cross-media retrieval method(CMSCR)with deep correlation analysis.By automatically exploring the semantic label,we can train the semantic mapping of the image-text maximum correlation subspace by crossmedia correlation learning,then we can obtain the image semantic space and the text semantic space.And then through the similarity measure method to calculate the correlation between text and images,in order to achieve cross-media retrieval.At the end of this thesis,we design and introduce the various aspects of crossmedia system,and classify and organize various popular algorithms of cross-media retrieval.In addition,a cross-media correlation search engine was designed and its user interface and retrieval results were presented through examples.