Semantic-based Video Retrieval Technology Research

With the rapid development of network and multimedia technologies and the increasing demand for multimedia information, video data is expanding rapidly. In such a vast information data, how to fast and effectively obtain the information which people need in the semantic level have become an urgent issue. Therefore, the semantic-based video retrieval has become a research hotspot. The main work is as follows:(1) A Fast slow feature analysis based shot boundary detection algorithm is proposed. Extract color histogram as the input signal of the slow feature analysis (SFA) algorithm, and implement shot segmentation by SFA algorithm. According to the problem that after the non-linear expansion,the dimension of input signal increases which may lead to an increase the complexity of the algorithm and the detection speed is slow, a non-linear dimension reduction method called local linear embedding (LLE) is applyed to achieve a fast shot boundary detection based on SFA.(2)An efficient visual attention based key frame extraction scheme is proposed. We extract the dynamic and static visual salience figure of the video frames,and fused by a adaptive weighted fusion scheme,then use the globally significant attention value to extract the key frame. For the dynamic attention model, temporal gradients are used to highlight the important areas of inter-frame motion.The static attention model is built using the image signature which employ discrete cosine transform based saliency detection.(3)A S-CTM (supervision correlated topic model) based semantic concept extraction model is proposed. According to the corresponding relationship between the latent topics and the target category is uncertain in correlated topic model(CTM), S-CTM utilizes class labels by enforcing a one-to-one correspondence between topics and class labels,which can reduce the uncertainty.The scale-invariant feature transform (SIFT) features of each keyframe is extracted,build a visual vocabulary by K-means clustering algorithm,and the S-CTM model is applied to extract the semantic concepts of keyframe images. (4)A clip based video retrieval method is proposed.Shot similarity is measured by the confidence of semantic concept of keyframe,and search many similar video clips preliminary. According to the problem that self-similar exists in the video clips, the maximum matching Hungarian algorithm is employed to filter out false similar video clips. Based on comprehensive consideration of four factor influence on video clips similarity,i.e.semantic similarity, temporal order, interference and granularity of shots, the total similarity of the video clip is obtained.