Semantic-based Video Retrieval

Content-based video retrieval (CBVR) is one of the most active hotspots in the fields of image engineering. It searches video data from large-scale video database by means of the content and context of video. People are accustomed to judge similarity using the high-level semantic concept, but the existing CBVR are mostly non-semantic level. Because of the semantic gap between low-level visual features and high-level semantic visual concepts, it faces enormous difficulties for video content description and manipulation at semantic conceptual level. Thus, bridging the semantic gap between low-level visual features and high-level semantic visual concepts and realizing semantic-based video retrieval may be the biggest challenge that we face in supporting semantic-based video retrieval. The main contents and contributions of this thesis are summarized as follows:(1) The key techniques of semantic-based video retrieval, which includes the video semantic extraction and the video semantic object extraction, are studied in depth. The presented techniques are summarized; also the advantages and disadvantages of them are analyzed from the view of application.(2) We propose an approach of automatic detection of spatio-temporal semantic object from video which is based on visual attention. It introduces visual attention and detects video semantic object in temporal and spatial domain separately. Then a dynamic fusion technique is applied to combine both the temporal and spatial saliency maps. Finally, video semantic objects are obtained. It is robust and accurate for the detection of video semantic object. The proposed technique is able to detect the moving semantic objects in video, as well as the objects which can intensely stimulate human's attention in the static video frame image. Experiment validates it higher performance and efficiency.(3) We propose a multi-level network video semantic extraction model. The model can be adapted with video semantic object which is extracted by our method. At the help of video semantic object as a middle layer, high-level video semantic concepts are obtained by correlation between levels. Eventually, semantic gap between the low-level features and high-level semantic concepts is solved.