Collaborative Video Summarization With Two Stage Sparse Optimization

With the availability of low-cost digital devices which are capable of high-volume video data,the growth of consumer videos is at a rapid rate.It can be a tedious and time-consuming work to process this video data manually.Therefore,how to automatically acquire the essential information of video data in the applications of video retrieval,sematic storage and interactive browsing,is a crucial problem that needs to be solved.Video summarization satisfies this need by extracting key frames which contain the essential content within the video.Compared with the existing methods that focus on news and sports videos,the summarization of consumer videos is more challenging due to its unconstrained content and the lack of edited video structures by human.To fulfill this challenging task,a method of collaborative video summarization with two stage sparse optimization is proposed in this thesis.In the first stage,reconstruction error of sparse coding is employed to select the frames which obtain the main information of the original video.Many existing methods have been developed to select representative frames by a dictionary learning model,which have led to a state-of-the-art performance.However,learning dictionary without considering relationship between samples of the original data space would lead to imprecise representation.To address this problem,in this thesis,geometrical distribution information of neighboring frames is incorporated into the dictionary learning process through a graph based learning strategy.After the first stage,a set of candidate key frames which contain the main content within video is generated.In the second stage,frames which represent diversity information within the video are selected.Since the video contain diversity information,the candidate key frames set can partition into multiple groups according to the dissimilarity content of frames.Thus given pairwise dissimilarities between the frames within the candidate key frames set,dissimilarity-based sparse subset selection is adopted to find the representative frames of all groups which contain dissimilarity content.The solution of the problem is treated as a row-sparsity regularized trace minimization problem.And the final video summarization is generated by these selected frames from the candidate key frames set.Through the above two stage,the video summarization generated by the proposed method could not only contain the essential information but also exclude the redundant contents of the video.To validate the effectiveness of the proposed method,experiments are performed on three independent consumer video databases,Kodak,CCV,EVVE.By comparing our methods with the state-of-art methods in details,the results demonstrate the proposed methods receive better performance.