Video Summarization Based On Semantic Preserving And Reinforcement Learning

The rapid development of information technology,video data shows explosive growth.A large number of video data contain redundant and repetitive information,which makes it more difficult for each user to obtain the required information quickly.In this case,video summarization technology comes into being,its goal is to generate a compact and comprehensive summary,in the shortest time to provide users with the maximum target video information,enable each user to obtain the required information more accurately and faster,improve the efficiency of users to obtain the required information and enhance the retrieval experience.Based on video summarization,this thesis studies deep attention model and reinforcement learning respectively.Firstly,the video summarization method with deep attention and semantic preserving is proposed to solve the problems that the existing video summarization algorithm ignored the internal relation between the original video and the abstract,semantic information loss and mean square error loss were sensitive to outliers,etc.The method by means of Bi-directional Long Short-Term Memory(Bi-LSTM)as the encoder to obtain the sequence of the original video information fully.In decoding part,an attention-based Long Short-Term Memory network is adopted fully explore the interrelation between the video summarization.Then,the semantic preserving module was introduced to minimize the loss of semantic information.Finally,Huber loss was introduced to mitigate the problem of the model's sensitivity to outliers,so as to generate the video abstract rich in information.The coordination of the various parts leads to the generation of informative video summarization.A large number of experiments on the benchmark data sets SumMe and TVSum verify the validity of the proposed model.Secondly,from the perspective of reinforcement learning,common video summarization algorithms seldom consider the representativeness and diversity of abstract generation at the same time.This thesis proposes an deep attention and reinforcement learning for video summarization algorithm,which defines video summarization as a continuous decision-making process,proposes an attention-based reinforcement learning framework,and designs a novel reward function to explain the diversity and representativeness of the generated summary.The depth and time domain information of video frames are obtained by using Convolutional Neural Network(CNN)and Recurrent Neural Network(RNN).The interrelation between video frames is explored through attention-based RNN so as to obtain representative and diverse video summarization.A large number of experiments on SumMe and TVSum benchmark data sets have verified the validity and feasibility of the proposed model.