Research Of Video Summarization Based On Multimodal Features Fusion

The development of multimedia and network technology makes video resources become more and more rich, and the accompanied problem is the rapid growth of video data. Therefore, how to process these massive video data effectively, thereby improving the browsing and retrieval efficiency has become a real problem should be solved. The technology of video summary can reduce the amount of video data, save browsing time and it is the key to solve these problems.News video is the main source for us to obtain information. Compared with the other video data, news video has its special structure and organizational characteristics, which make the summary of the news video technology, become the hot spot of extensive research. This thesis focuses on this topic and has made some achievements.(1) Video shot segmentation. A shot boundary detection algorithm based on adaptive threshold is proposed. On the analysis of the traditional shot detection methods, and according to the characteristics of most of the shots are cut shot and often exist flash events, shot boundary initially identified by secondary detection and adaptive threshold adjustment, and then add the conditions to judge the flash events, which purpose is to filter the flash events, and ultimately determine the shot boundary. Experiments show that the adaptability of the flash in this algorithm increases greatly, and improve the shots detection accuracy significantly.(2) Detection of anchorperson shots. An anchorperson shots detection algorithm based on audio and video features is proposed in this paper. Anchorperson template is automatically extracted by considering the characteristics of the mute fragment. And then use the characteristics of the same background to detect the anchorperson shot through the calculation of the color distance and template matching. This method is fully automatic and there is no human-computer interaction, be advantageous at high accuracy and adaptability.(3) Key-frame extraction technology. Propose a method of key-frame extraction based on the fusion of shots and the closed-captions. According to the particularity of the news video, we select two kinds of frames as key-frames:one is the subtitle which contains the news topics, second is the frame which is closest to the midpoint of time in each shot. The experimental results show that the key-frames extracted by this method have a better representation and better description of the content of the news video as well. (4) Method for generation and forms of video summary. Considering the video’s audio and video information, we proposed a method for news video summary generation based on multi-modal features fusion, including a static summary of the news story board, video summary based on the compression of different ratios, based on the anchorperson shots or based on the detection of closed-captions in news video.Finally, design and implement a prototype system of news video summary based on multimodal features fusion, which integrates the main results of this paper. Test results show that users are relatively satisfied with the summary results.