Cross-Media Feature Learning With Semi-supervised Graph Regularization

In recent years,with the development of Internet technology,multimedia data on the Internet has exploded,gradually transforming from a media type to various media types such as text,image,video,audio,3D model,and so on.Blended together to present richer and more comprehensive semantic information.However,the traditional single-media retrieval is difficult to meet the needs of people to search for different media types.Therefore,a new information retrieval technology-cross-media retrieval emerges as the times require,and it has quickly become a hot topic for scholars at home and abroad in recent years.Cross-media retrieval technology breaks through the limitation that single-media retrieval is limited to retrieving data of the same media type.The retrieval mode is to retrieve data of other types as related data objects by using one type of media data,such as using images to retrieve semantically similar texts,audio data,etc.The key challenge of cross-media retrieval is to solve the “semantic gap” between multimedia data,that is,data of different media types have different underlying features for the same high-level semantics.Therefore,how to accurately measure the semantic similarity between them and then carry out cross-media feature learning is very challenging.In order to solve the above problems,this paper conducts cross-media feature learning based on semi-supervised graph regularization,and applies graph regularization to protect the similarity between multimedia data.The similarity matrix is used in the graph to clearly measure the relationship between data.On the basis of this,cross-media patches are applied to highlight important parts of multimedia data,thereby improving the accuracy of cross-media relevance.In this paper,the mean average precision and precision-recall curves are used as evaluation criteria to conduct experiments on different cross-media datasets,and the experimental results show the effectiveness and superiority of the proposed method in cross-media retrieval.