Research On Video Summarization Based On Multi Scale Temporal Information

Video summarization technology extracts key frames or important segments with a limited number to generate static key frame sets or dynamic visual summaries,providing an effective method for quickly obtaining the core content of the video.With the explosive growth of video data,video summarization technology has good research significance and application scenarios.This article analyzes and explores the different aspects of video abstract generation.The main research work is as follows:(1)A feature extraction algorithm based on self-supervised learning is proposed.The model automatically generates high-confidence label information for unlabeled image sample,and expands the size of data set to improve its performance.Hard-negative data is trained iteratively through sample mining,and videos are tracked with multi-target detection.The algorithm we proposed uses self-supervised learning to acquire the sample distribution of video data,extracts image features without relying on manual annotation.It can effectively use the static visual features and dynamic sequence information of massive videos.(2)A key frame selection algorithm based on multi-scale timing information perception is proposed.The model is composed of multi-level feature fusion module and multi-branch correlation analysis module.The algorithm obtains visual and semantic information according to the interframe dependence and sequence dependence in different scales,calculates the importance score of the video frame,and selects the key frame set that can characterize the core content of the video.Using local feature fusion and global correlation analysis of time series information,the performance of our algorithm is comparable to the best existing methods.(3)A video summary generation algorithm based on unsupervised learning is proposed.We designed two video summarization models,using reinforcement learning and generative adversarial networks respectively.Unsupervised learning provides an objective and effective evaluation criteria of video summary.The experiment compares the quality of video summaries generated by different unsupervised learning methods and supervised learning methods.The results show that our proposed unsupervised learning video summarization model can adaptively generate high-quality video summary,and its performance is better than most current supervised learning methods.