Visual Semantic Detection Based Video Content Indexing

Video semantic indexing based on visual concept detection is used to de-tect video semantics, which is the fundamental step for the arranging, filtering, classifying and indexing of the huge amount of multimedia corpus. Video se-mantic indexing has a huge research area and economical potential.This paper addressed the main algorithms and frameworks about semantic indexing based on visual concept detection and the innovations. The innova-tions are listed in details,From video to image-video summarization based on visual similarity. The combination of MSLBP and normalized Euclidean distance is used. In the clustering model selection part, AHC based clustering are used to gather video frames into clusters. With the Interesting Function which are consist of video structure, audio and faces to determine the summarization frames, video streams are transformed into a subset of frames.From image to labels-Content based image classification. Frames from video summarization are used to do image classification. After a detailed s-tudy of features and descriptors, TransformedColorSIFT and VLAD model are introduced before dimension reducing step by Kernel-PCA. Images are then transformed into vectors. SVM based classifiers are trained and tested to get the right labels. Multiple early fusion and late fusion methods are used.In the experiments part, video summarization has achieved a precision of70%to96.7%, what’s more, with the help of AHC clustering model, various stages of summarization could be achieved. For the content-based image classi-fication framework, TRECVID2012semantic indexing tasks were introduced. Our system took part in that task and achieved top10of100participants, which proved the robustness of our algorithm and the feasibility of our system. In the further study, we will take our effort to meet the challenge of video indexing via smart phones and Pads, which means more compressed features should be developed and the computation cost of whole system should be opti-mized considering the hardware of those devices.