Key Technologies On Cross-Media Retrieval

With the continuous updating of multimedia devices and the rapid development of mobile application technology, multimedia, especially videos spread more widely, people’s requirements for video retrieval system are also growing. The main work of this paper is to build cross-media video retrieval framework, which includes semantic extraction of videos, video clustering based on scenes etc. The main contributions are:1. Propose a framework for cross-media search, and the search task could be executed by images, audio and texts. For the preprocessing of videos’metadata and query text, we complete spell checking, stop words dropping, stemming and lemmatization and so on. Secondly, as to the videos’semantic information extraction, we extract the audio, text, and face, some objects, and other concepts of images to represent video content. Indexings of video metadata and videos’ semantic information are established by lucene separately. Experimental result on TRECVID-KIS dataset shows that the search MAP is0.265, which verify the effectiveness of proposed framework.2. Propose a stroke-based text detection and OCR algorithm, which help the semantic video understanding. The algorithm is applied to cross-media video retrieval and improves search performance.3. Propose a scenario-based video clustering algorithm. By analyzing the scene and activities of the video, we extract a higher level of semantic information of videos. Firstly, BoW model is used to quantize videos into visual words, and then spectral clustering algorithm is applied to cluster videos into different classes with the same topic. Experimental result on self-built network video dataset of hot events shows that the proposed method can improve the accuracy of video retrieval.