Video Relevance Feedback Retrieval System Based On Semantic Annotation

Since the 1990s, Content-Base Video Retrieval has become a hot research topic. In the era of information explosion, it has become an urgent problem in the research of multimedia that how to make computer to get semantic information from videos automatically and using them to retrieval videos efficiently. The advantage of semantic representation of video is that the recognition based on semantic accords with human's cognition understanding. Using semantic representation is an effective resolution of Semantic Gap between low-level vision features and high-level concept features. In this paper, our objective is to propose a video retrieval method based on semantic. It relies on the automatic annotation information for videos. And use a relevance feedback method with long-term memory to retrieval video semantic.Relevance feedback is an effective method of improving the retrieval accuracy using man-machine interaction. A method of relevance feedback based on Support Vector Machine (SVM) is discussed and implemented in this paper. After a retrieval process, user submit the correct results as positive examples and retrieval system select the negative examples according to amount of positive examples and rank information of retrieval results. These two class examples co-construct a training data set for SVM. After obtaining a SVM classification model, the retrieval system runs a new classification retrieval process with this model. Generally, the training set from user feedback is a small sample set. A SVM classifier trained from small sample set has excellence performance on classification according to the previous work. In addition, a fast algorithm, Sequential Minimal Optimization (SMO), is used to speed up the training process of SVM.Relevance feedback is effectively, but it can not remain the feedback information for a long time, hence its short-term memory capacity. Furthermore, Semantic Gap must have adverse effect on a feedback system based on low-level vision features. The paper proposes a video semantic retrieval system with long-term relevance feedback. First, a low-dimension semantic feature is constructed. And then an associated network between key frames and semantic concepts is constructed. This associated network can be modified during user's feedback operation. And user's feedback information can be remained in a long term. Finally, the system can modified the associated network and return the key frames which are closely related to the query. The model of video retrieval proposed in the paper can remain the long-term information spanning different queries. It can improve the performance of retrieval using the accumulated knowledge. Our retrieval system can match users' intent better because of using the semantic feature.