A Video Description Scheme For Fast Video Previewing And Its Implementation

With the spread of the Internet and the change of the form of entertainment,watching online videos is nowadays an indispensable part of one's daily life.In recent years,the number of online videos has been growing rapidly.It is crucial to generate overviews of videos to help audiences make viewing decisions and save time.Common solutions include video summarization and video captioning.However,the overview generates from video summarization still costs quite long time,while the one generates from video captioning lacks meaningful understanding of the content.We made researches on solutions to video overviewing and related visual techniques.Using multiple techniques,a video description scheme that auto-generate video overviews is proposed.The proposed scheme mainly focuses on human-centered videos with certain story backgrounds.It consists of two stages:video steam preprocessing and video overview generating.Visual information such as detection boxes of bodies and faces,tracklets and shots are extracted in the first stage.In the second stage,scenes,individuals and the final overviews are extracted and generated according to the information from the first stage.Some methods in the scheme are selected from the existing studies through comparison,while the others were first proposed in this paper.Specifically,the proposed face clustering method and scene detection method outperform other state-of-the-arts methods and are highly generalizable.To demonstrate the experimental results and usage of the proposed video description scheme,a demo application was developed.The main functions include uploading and searching videos,playing special scenes,browsing video overviews and managing characters and scenes.The video overview generated from the proposed scheme is generalizable and quicker-to-watch.It can help users of online video websites save their time when sifting through contents.The main contributions of the paper are as follows:(1)A new video description scheme was proposed to generate overviews of online videos which give the users a quick glance at the content.The content of the original video is transformed into person-scene pairs.If there is no person in a scene,then only the picture of the scene will be displayed.Compared to video summarization and video captioning,the new scheme is much briefer and is highly generalizable.(2)A new face clustering method was proposed to identify individuals in a video.This method is based on prudent label propagation.With consistency of tracklets of people utilized,the iteration of the labels is based on tracklets.The experimental results on the dataset of real-world videos show that the method outperforms other state-of-the-art methods.(3)A new scene detection method was proposed to divide a video into scenes.The method uses both visual and audio info rmation and divide the scene detection task into spatial clustering,temporal clustering,logical clustering and audio clustering.The experimental results show that our method improves the detection result from 3 perspectives and is highly generalizable.(4)A web application was developed to demonstrate the usage of the proposed description scheme.The application can be used to show overviews of different types of videos generated from the proposed video description scheme and demonstrate how to take advantage of the scheme.