Research On Automatic Summarization Of Time-sync Video Comments

The rapid growth of online sharing media brings a huge collection of videos.In the meantime,the volume and complexity of video data pose a challenge to video anno-tation.To address that,automatic video tagging techniques have been proposed which mainly focus on detecting concepts and associating tags of entire videos.However,tra-ditional methods for automatic time-sync description of video content usually rely on scene-text alignment,which can be very expensive in practice.Fortunately,time-sync video comments provide a valuable source of information regarding the temporal information of a video,which offer a new perspective for video analysis.In this thesis,we investigate how to exploit the information of time-sync comments to generate temporal descriptions of videos.The challenge of the proposed application comes from the extremely noisy nature of the comments,which are usually on very different topics.To resolve these issues,we propose a novel temporal summa-rization model based on the data reconstruction principle,where representative com-ments are selected in order to best reconstruct the original corpus at the text level as well as the topic level while incorporating the temporal correlations of the comments.As a result,the summary provides an informative description of the video in a time-sync manner.The main contributions of this work can be summarized below:(1)We propose a novel method that generates temporal descriptions of videos by summarizing time-sync comments.(2)The model integrates text and topic reconstruction simultaneously to resolve the issues of noisy and redundant information contained in the time-sync com-ments.In addition,we also consider the temporal nature of the problem with dynamic summarization of comments in sequential segments.(3)Experimental results on real-world data demonstrate the effectiveness of the proposed framework and justify the idea of exploiting time-sync comments as a bridge to describe videos.