CNN-based Shot Boundary Detection And Video Annotation

With the development of the digital media and network technologies,the digital video information has an explosive growth.As a result,effective analysis and management of the massive video resources has draw a lot of attention.Shot boundary detection and video annotation are two important steps of the video analysis technology.The accuracy and speed of the two procedures directly influence the performance of the video analysis.Recently,many researchers have studied the shot boundary detection and proposed their algorithms.However,those methods have drawbacks in detecting gradual transitions.For instance,some of them don't meet the accuracy and others have too high computational cost.This article proposes a shot boundary detection method based on candidate segment selection and CNN(convolutional neural network),which achieves good performance in detecting gradual transitions.The main contribution of this article on shot boundary detection is:We optimize the candidate segment selection in the feature extraction,adaptive thresholds and biosection comparison procedures to get candidate segments which contain shot boundaries can be detected as more as possible.CNN is also used to extracted the high level features from the static frames.We use those high level features to judge if candidate segments contain shot boundaries in them.Experiment results show that the proposed scheme can achieve a high detection speed and an excellent accuracy compared with other methods.Video annotation is considered to be a good way to fill the semantic gap between the semantic information of the video and the low level features.It attaches segment labels to the video shots based on the video contents.Thus,it is an essential part of the content-based video retrieval.The traditional video annotation methods rely on the artificial feature extraction methods(HOG,SIFT,and so on)too much.The quality and quantity of features become a bottleneck for the video annotation.Motivated by that CNN can study multi-layers features,we apply deep learning into the video annotation to improve the performance.The main contribution of this article on shot boundary detection is:We propose a CNN-based video annotation algorithm framework.We also study how to transfer the large and deep CNN structure to a new dataset.In this article,four methods are used to implement the transfer: 1.The method based on lexical semantic similarity.2.Training a new network.3.The fine tuning of a deep network.4.Partly fine-tuning of a deep network.Experiment results show that the proposed annotation scheme has a high promotion compared the state-of-the-art method.