Video Keyframe Extraction And Frame Rate Up Conversion Based On Vision Saliency

Video saliency detection has been a hot research on computer vision. Based on saliency, this paper summaries and extends two video process methods- keyframe extraction method and frame rate up conversion.Keyframe extraction method provides an effective means for fast retrieval of video content. With the continuous requirements of social security and monitoring equipment widely used, a large amount of video redundancy data has been produced, which makes retrieval of interested content of video necessary. The frame rate up conversion of video can improve the original video with upper frame rate and get a better effect on viewers. After elaboration states the basic concepts and principle of the keyframe extraction and frame rate up conversion, a keyframe extraction method based on construction the saliency model of the moving target in the surveillance video and a frame rate up conversion method fusion with video spatiotemporal saliency proposed. The details are as follows:(1) A new keyframe extraction method suitable for surveillance video is presented, the main work can be seen as follows: First, constructing a new visual attention model for the moving target in surveillance video, the model consisting with feature maps about color, texture and shape of object. Skin confidence map is also obtained by skin model. Then three feature maps and skin confidence map are dynamic weighted combination into a multi-feature map that includes only true attention regions. Combination of a variety of feature can overcome the shortcoming of the single feature can't describe the characteristics information completely. Second, since the method towards moving target in the surveillance video, the extraction frames about targets can provide effective processing samples for video post-processing,such as face super-resolution reconstruction. Results show that the proposed keyframe extraction method can grasp the pedestrians' information quickly, making fast retrieval of surveillance video possible, and will offer effective samples of objects for video post-processing.(2) A new improved method about frame rate up conversion is presented fusion with video spatiotemporal saliency. The method can get the motion vector of the region which interested people more precisely. Compared with the original DME algorithm, concrete improvement takes dynamic textures to measure the saliency of the block to be processed, and divides the blocks into two groups by a threshold value preset. For the first group, the motion vector is processed by vector refinement(MVR) and with the other one processed by vector consistency(MVC). The experimental results show that the candidate motion vector is more accurate than results processed by other methods.