Decoder Oriented Visual Attention Model For Video Summarization

With the rapid development and popularity of mobile internet technology and multimedia video capture devices,massive video datas brings great pressure to video retrieval,video surveillance and video archiving applications.Video summarization has drawn wide attention as a way to quickly browse and understand video content.We conduct thorough research on video summarization in terms of supervised method and deep learning.In this paper,we propose a decoder oriented visual attention model for supervised video summarization,which combine the existing encoder-decoder framework with a novel visual attention model.Different from the previous attention model which focus on the encoding sequence,we pay attention to decoding sequence by using Long ShortTerm Memory network.This considers intrinsic association between video frames using attention model.Utilizing the previous decoding sequences could guide the current decoding process effectively,which improves the accuracy of our model's prediction.The proposed algorithm is mainly divided into four steps: video segmentation,video feature extraction,importance measure of video shots and keyshots selection.Specifically,KTS algorithm is used to segment video shots,and video frames features are extracted through the pre-trained Convolution neural network,then using the attention model to predict the importance scores of video shots,and finally the dynamic programming method is used to generate video summary.Extensive experiment is conducted on SumMe,TVSum,YouTube and OVP datasets.Compared with other algorithms,our experimental results clearly demonstrate the effectiveness and superiority of he proposed method.