Semantic Based Cross-Media Retrieval

With the rapid development of Internet, computers and multimedia technologies, how to quickly search and effectively manage vast amounts of multimedia has become an urgent problem to be solved and a research hotspot in recent years. This paper lays research on text-based cross-media semantic search, video text detection and video source identification, and puts forward some new algorithms and frameworks.1. In the semantic-based cross-media search, this paper proposes a semantic based video retrieval framework which search for indexes from different media according to the query. The system is divided into two parts, one is text-based framework for semantic video search and the other is image-based audio and other media semantics video search. The framework realizes text analysis for pre-treatment, the extraction of the semantic text and video content, semantic concept detection of subtitles, audio, video, and other media sources, and uses Lucene to complete the retrieval and sorting. The system is centralized in a semantic-based video search task (KIS) test TRECVID2012tests and ranks fourth, which fully reflects the effectiveness of the proposed framework.2. In extraction and semantic analysis, compared to other detection of video semantic concept, video has an important role for a direct understanding of the text content. The reason is the text appears in the video, especially the title and subtitle, is a kind of video content or high-level semantic description that is very effective to detect the contents of the entire video semantic description more immediacy. For the text of the video detection and identification, detection method is proposed and implemented with a static video subtitles and scrolling marquee extracted corner and visual features, such as optical flow, training. The method uses Support Vector Machine (SVM) as a classifier for the detection and extraction of video text and subtitles, while after treatment uses Optical Character Recognition(OCR) software to identify. The algorithm is tested in the dataset to prove its effectiveness.3. In semantic extraction and analysis, for the existing mainly for scenes, objects, and other video or image content semantic concept detector, this paper proposes and implements a method for detecting the source of the video source, focusing on the video source identification -sets target recognition algorithm. Two-stage cascade search strategy, use the edge of the template, RGB color space histogram, histogram of oriented gradients, improved γ-LBP as a visual feature, using the nearest neighbor similarity search method step by step to calculate the image and the template logo area. Self-test datasets demonstrate the effectiveness of this algorithm. This detection of the video source provenance can effectively limit and reduce the scope of the search, which has a direct impact on the efficiency of video search.