Research On Video Abstract Based On Layered Representative Frame Extraction

With the wide use and continuous improvement of video capture, storage devices and network transmission performance, digital video industry, supported by state, has developed rapidly in television, Internet and mobile phone. Faced with such a huge variety of video, people are looking forward to the emergence of a rapid and efficient selection method to save their time and efforts. As a streamlined technology to reflect the main content of videos, video abstract technology, can solve the "difficult choice" problem of interesting videos.This thesis mainly studies static video summary based on low-level characteristics, figure objects and speech events. The core technology is the representative-frame extraction. The main work that has been done is described as following:Firstly, starting from color feature of video frames, the traditional color coherence vector(CCV) algorithm is improved since it can't accurately describe video frame characteristics due to fixed-connectivity threshold and the fixed cut threshold in the equivalent relation based clustering can't adapt to changes in rich video content. In addition, the temporal feature is added for generating the video summary, used to extract representative-frame more accurately. The process doesn't need human intervention and the experimental results with adaptive threshold are satisfactory.Secondly, the conception and extraction algorithm of "figure-based frame" is introduced and Gaussian mixture model is trained in "mixed color space", which not only greatly reduces the number of detection frames of figures, but also increases the description accuracy of the skin collective characteristics. Video summary of figure object generated by detecting the figure on figure-based frames meets user's rapid understanding of figures in videos.Thirdly, combined the moving region with the face detection, video summary of speech event is generated by extracting frames of speech events. The corresponding relationship between figure-based frames and their moving regions is proposed during moving regions extraction. When using Haar-like feature and AdaBoost algorithm to detect the face, the problems of large number of rectangle features and multi-scale face is solved by ignoring smaller rectangles and changing the size of the detection window, the specific expression of rectangle feature values is given, and the ordered sub-threshold method is employed for every weak classifier which can choose the weakest classifiers more accurately. The generated video summary of speech event reflects figure's movement and behavior in videos.