Research On Graph Model Based Video Summarization

In recent years,there has been an explosive growth of various videos on the Internet with the proliferation of multimedia and network technologies.It is increasingly necessary to quickly browse and view massive video data in a limited time to facilitate user video browsing and video retrieval.In addition,the surveillance systems used for security produce a large amount of surveillance video data every day,and such a huge amount of data brings many difficulties to video data storage and archiving.Video summarization(VS)techniques to generate a simplified version of a video stream that retain only their most informative and representative content have become an effective solution that is receiving increasing attention from researchers.In general,shot boundary detection-based VS methods are implemented by analyzing inter-frame differences,where significant differences indicate the possible existence of boundaries at the detected locations.Shot detection algorithms generally include steps such as video representation,inter-frame difference analysis,and decision methods.Suitable video representation and inter-frame difference analysis methods are the key and challenges that constrain the performance of such methods.Currently,there are several drawbacks of such methods,mainly include:(1)The direct use of low-level features such as pixels,pixel blocks,histograms,and descriptors to represent video lacks practicality.Direct video representation based on such features has proven to be effective in detecting sudden shot changes(e.g.,hard cutting).However,its main limitation is the lack of ability to detect subtle transitions(fades).(2)The use of a single dissimilarity metric lacks universality.Although various metrics on the video shot boundary detection show certain validity,they have different detection effects for different shot boundaries.Therefore,for a given video,which metric is more important for the effective detection of shot boundaries is sometimes elusive,mainly depending on the specific video content structure.(3)The use of a fixed threshold strategy in detecting shot boundaries lacks flexibility.The fixed threshold strategy does not consider the video dynamic and ignores the high variability of inter-frame differences,which increases the false detection rate of shot boundary detection.In this paper,a series of studies are conducted to address the above issues,and the main work and contributions are as follows.(1)To address the lack of practicality of traditional low-level video features to represent the video,a video representation method based on graph modelling is proposed.Considering the structural information of video frame features,graph models are constructed to better represent frames to bridge the gap between the actual semantic structural information and the low-level features of video frames.The detailed structural changes between graphs are more consistent with the actual changes between video frames.Therefore,the newly proposed method is robust to detect various types of shot transitions,such as hard cuts,dissolves,wipes,fade-ins/fade-outs,etc.,laying the foundation for the subsequent research content.(2)To solve the problem that a single dissimilarity metric lacks universality,we propose an adaptive weighted metric design for graph dissimilarity metric.Four graph structure-oriented distance metrics are analyzed,and the detection advantages of the four graph dissimilarity metrics are adaptively fused to learn a more accurate and reliable dissimilarity metric for calculating inter-frame differences,which solves the problem of being limited to a single dissimilarity metric and enhances the universality,robustness and accuracy of detecting various types of shot transitions.(3)To address the lack of flexibility in detecting shot boundaries using a fixed threshold strategy,an adaptive threshold-based strategy is proposed to detect shot boundaries.Considering the video dynamic,the 3-sigma rule following a typical Gaussian distribution detects potential shot boundaries,solves the problem of being limited by a fixed threshold,and improves the robustness and accuracy of detecting various types of shot transitions.