Research On The Key Frame Extraction Algorithm On Video Semantic Detection

With the rise of live-broadcast platforms and the development of the Internet,video has become a major way for people to generate and obtain information.How to deal with these massive data and realize IVA(Intelligent Video Analysis)has gradually at-tracted people's attention.As the foundation of IVA,video semantic detection has very important research significance and value.Due to the large amount of video data,most research on video semantic detection is based on the keyframe extraction.Therefore,it is also very necessary and critical to do some research on keyframe extraction technol-ogy.This thesis focuses on the research of keyframe extraction and semantic detection of video,mainly from the following three aspects:First,we study the keyframe extraction algorithm based on shot boundary detec-tion.After analyzing and comparing several classical algorithms of shot boundary de-tection,an improved dual threshold method is proposed.We add a weight template to the color-histogram,and use a local adaptive threshold instead of a fixed threshold to avoid the influence of manual fixed threshold.After segmenting the obtained shot into sub-shots,we use the difference value of each frame to judge frame differences and extract keyframes in each sub-shot.Because present keyframe extraction algorithms have the problem of losing orig-inal video information,we introduce the video synopsis technology into keyframe ex-traction.After studying the principle of video synopsis algorithm and building a map-ping model from original video to concentrated video under some constraint conditions,we define the fitness function and get the concentrated video by solving it.Compared with the traditional key frame extraction algorithm,the video synopsis technology not only removes lots of redundance and obtain higher density frames,but also retains more information in the original video,especially time information and dynamic information.Finally,based on the study of the basic theory of conditional random field,we do some detailed theoretical research and derivation on the conditional probability model of the characteristics of low-level video images and the conditional probability model of the mapping between pixels and objects.According to the image feature model,the conditional probability model of image features and the mapping model between pixels and objects,the conditional probability model of object semantics is deduced.Finally,three kinds of keyframe extraction algorithms are tested and compared on this model.The experiment shows that the detection results of the improved keyframe extraction method based on shot boundary detection and the keyframe extraction method based on video synopsis are better than the traditional keyframe extraction method based on dual threshold shot segmentation.