Research And Implementation, Of Cross-media Similarity Mechanism

In modern society, the amount of multimedia data is increasing dramatically, which makes people's information source extends to multiple modalities of media objects, such as image, video, audio, 3D and flash. Meanwhile, the coexistence of these media objects shows a cross-media feature, characterized as the mixed coexistence of multiple types of media objects, complex structure among media objects, and the cooperation of different types of media objects to express a common concept. This cross-media character between media objects obliges us to take it into consideration during multimedia retrieval, and to make proper use of it. That is, we need to fuse multiple modalities of media objects to mine the latent semantic relationships among them, and then we can cross from one type of media objects to another via semantics to realize cross-media retrieval. However, one of the first problems in cross-media retrieval is the similarity measurement of different types of media objects. Under this background, we present two cross-media retrieval algorithms in this paper, focusing on similarity measurement. One uses similarity fusion and the other employs PageR-ank. This is our first attempt in the area of cross-media retrieval.In this paper, first, we will introduce our research background, as well as the current technology and their deficiency in this area. We will also give a quick look of this paper's main contribution.In chapter two, we will review the history of multimedia analysis and retrieval, mainly about the progress in the area of text retrieval, keyword-based multimedia retrieval, content-based multimedia retrieval, automatic multimedia annotation, and multimedia fusion and understandingWe then present a cross-media retrieval algorithm based on similarity fusion in chapter three. We first fuse the similarity between different media objects via Sim-Fusion algorithm, and then by doing MDS, we can obtain the media objects' coordinates in semantic space. Using these coordinates, we can realize cross-media retrieval.In chapter four, we introduce another cross-media retrieval algorithm, which is based on Personalized PageRank and cross-reference graph. We improve the Personalized PageRank and apply it to the cross-reference graph, so that we can rank the media objects according to their similarity with the query, just as PageRank do according to webpages' importance. The result of this algorithm is satisfying.At the end of this paper, we give the conclusion and future work.