Video Structure Analysis Based On Face Clustering

Multimedia data is different from text data, which can take advantages of literal information such as index, abstraction, keywords. There is no transition level between the whole video file, video's biggest granular, and the individual frame, video's smallest granular. The traditional way of video browsing is play sequentially according to timestamp of each frame. By this method, it occupies too much of user's time before he finds his interest part. Its lack of intelligent skipping make it a bad experience to users, especially in today's video sharing websites. Based on story unit, short video clip is much popular in next generation of video content service. In order to generate short video clips from video streams of broadcast program, in this paper, we introduce a video structure analysis method based on face clustering. Comparing with traditional ones, our method has some innovations listed below:Designed a system compose of automatic shot segmentation, keyframe extraction and face detection, similarity of faces calculation and video structure analysis after un-supervised face clustering. This system is the important part of backend engine in our video browsing website, consuming every entire video program and outputting structure results file, which enable structure visualization and intelligent skipping.Gabor feature is a overall description of face image while the SIFT is the local interest point. Integrating these two types of feature, our method to calculate similarity of face can handle the worse condition of face image, like pose, illumination, and expression. In the dataset of LFW (Label Face in Wild), which focus on the un-control condition of face verification, our approach achieve 0.7927 average precision.Base on graph-theoretic face image clustering, images of the same semantic concept-the same person in different time stamp are group together, shots are group together accordingly. This time distribute among a certain shot set can be use to determine the anchor person shots. In the data set of 16videos from 3 French TV channel, experiment shows that our anchor detection method achieve 93.205 F-measure.