Information Fusion Based Multimedia Content Search

The goal of content-based multimedia information search is to quickly and pre-cisely return the multimedia data needed by users from a huge multimedia database. Currently,existing methods for multimedia search are mainly feature-centered,which evaluate the relevance between multimedia and user's query intent via similarity mea-surement among feaItures of multimedia data. However,it is difficult to achieve an expected search performance due to the rich content of multimedia and the well-known semantic gap between low-level features and user's high-level semantics.Hence,it is necessary to develop some methods to effectively fuse multiple aspects of multimedia information so as to bridge the semantic gap and improve the search capability.In this thesis,we investigate the problems existing in current multimedia search systems in depth and propose some feasible solutions from the aspect of content analysis and fusion.The main contributions are as follows:1.We propose an interactive multimedia search method.which is baseed on a multi_view cooperative learning strategy.This method can automatically increase feed-back information by cooperatively inferring positive samples from initial search regults against the multi-view information.In essence.this method is a kind of semi-supervised interactive search methods,which can tiger the automatic infer-ring mechanism given only a very few positive samples labeled by users.In this way,the lbeling burden on users can be greatly alleviated. The experimental results show thalt this method not only significantly reduces the labeling efforts on users but also achieves improved search precision,especially on the top ranked results.2.We propose a video reranking method,which is based on the multi-modal in-formation fusion. This method takes user satisfaction and user behavior into account when reranking the initial search results. Using the information from multiple modal spaces,this method Cooperaltively inflers the most relevant results and ranks them to the top of list,leading to an improved result list.The exper-imental results show that this method indeed improves the search precision of results on the top of result list,especially on the top 30 results.3.We propose a high-level concept detection method,which is baged on.the fusion of semantic correlation information among Concepts.This method first constructs the semantic correlation among concepts against ontology,and then improves the detection performance of traditional detectors by effectively fusing the correlation information.The experimental results show that the detectors which fuse the correlation information indeed perform better than the traditionaL detectors.4.We propose a frame fusion based copy detection method.This method Can ef-fectively handle some complex video transformations in copy by replacing strict temporal consistency assumption with relaxed state transition constraint and emission constraint.In addition,an additional gap constraint is introduced for determining the starting and ending positions of copies. By combining three constraints into a Viterbi-Like algorithm,the proposed frame fusion method can flexibly and efficiently deal with copy detection problem in a continuous query video stream.The experimental results show that this method not only over-comes the detection problem of long video stream in traditional methods but also achieves an excellent localization performance.