Multi-video Summarization Based On Graph Model

With the tremendous growth of web videos,it is more complicated for users to capture accurate information quickly and manage the video content of their interest,so that the intensity of work is getting bigger.Video Summarization is a good solution to solve this problem and it can perceive important content of videos quickly.Traditional video summarization methods based graph usually consider video frames as vertices to construct simple graph models.Although it has better performance,it can not represent the complex relationship of video frames.Since the content of multi-video data sets are diversity,redundancy and have large noise,complex graph models are needed to better characterize video content.To solve this problem,we apply hypergraph to describe complex relation-ships between video frames.In addition,multi-modal graph is also used to model the relationship between video frames with the help of tag information of video and complementary information of web images.These two graph models are applied to improve Multi-Video Summarization(MVS)algorithm.To well address the challenges in MVS,we firstly propose a novel framework with Hypergraph Dominant Set(HDS),termed MVS-HDS.Specifically,we first formulate the MVS task into a problem of finding dominant sets in a hypergraph to select the candidate frames.Then,with the help of web images searched by the same query,a Query Dependent Maximum Marginal Relevance(QD-MMR)method is developed to refine these candidate frames by balancing the criteria of redundancy and query adaptation.In order to make the summarization user-friendly and easy to understand,we also propose a Graph-based Topical Closeness(GTC)method to make it more comprehensible.In addition,we propose a Multi-modal Weighted Archetypal Analysis(MVS-MWAA)method to extract a concise summarization which is both representative and informative.In order to overcome the redundancy,diversity and other issues of multi-video data sets,we establish a multi-modal graph to guide the generation of the weight in WAA,which we call query-dependency weight.Specifically,the multi-modal graph fuses the complementary property of video frames,tags,and query-dependency web images.The candidate frames obtained by shot detection are divided into different archetype sets and the summarization is selected according to the importance scores of the archetype sets and the video frames in the archetype sets.Experiments for both approaches are carried out on two multi-video data sets of MVS1 K and TVSum.The experiments results demonstrate that our proposed methods clearly outperform the state-of-art approaches,which verify the effectiveness and superiority of both proposed algorithms.