Research On Content-Aware Video Stabilization And Retargeting

The main research topics of this thesis comprise four points:1)saliency map Detec-tion;2)video stabilization;3)content-aware video retargeting;4)exemplar based texture synthesis. The chapters from chapter2to chapter5are corresponding to these four research topics.In the second chapter, we propose a random saliency map detection algorithm. Most image retargeting algorithms rely heavily on valid saliency map detection to proceed. However, the inefficiency of high quality saliency map detection severely restricts the application of these image retargeting methods. In this paper, we propose a random algorithm for efficient context-aware saliency map detection. Our method is a multiple level saliency map detection algorithm that integrates multiple level coarse saliency maps into the resulting saliency map and selectively updates unreliable regions of the saliency map to refine, detection results. Because; of the randomized search, our method requires very little additional memory beyond that for the input image and result map, and does not need to build auxiliary data structures to accelerate the saliency map detection. We have implemented our algorithm on a GPU and demonstrated the performance there of on a variety of images and video sequences, compared with state-of-the-art image processing.In the third chapter, we introduce a novel2D-3D hybrid video stabilization method which combines virtues of2D and3D video stabilization methods in one routine. It at-tempts to achieve high-quality camera motions and to retain full frame coherence in each frame, at while, ensure that local regions undergo a similarity transformation. Compared with2D video stabilization,3D video stabilization can achieve more stable camera motion, even provide the capability of planning camera trajectory. But when casual video is very jittery or motion object in the scene is very large,3D reconstruction will be error-prone and overkill for the stabilization problem.2D video stabilization esti-mates camera motion based on pairwise registration between frames to compute motion compensation. It is robust and efficient, but the motion model of2D video stabilization is weak, and therefor the amount of stabilization it can provide is limited. We solve the stabilization problem by integrating3D and2D video stabilization methods into one routine. It smooths camera motions and explicitly employs local motion information which constraints video frames to be temporal coherent, and achieves high-quality video stabilization. Experiments show that our method not only can achieve high-quality cam-era motion on good3D reconstructed scene, but also can deal with complicated videos containing near, large moving objects.In the fourth chapter, we present an integrated approach which can solve video stabilization and retargeting at one time. Both video stabilization and retargeting are important post processing techniques for film production. In many cases, we need stabi-lize the shaky handhold video and adapt it to target displays with different resolutions and aspect ratios. Video stabilization and retargeting are computationally costly and both require a large amount of memory. If we stabilize and retarget a video serially, the time cost will be the sum of two separate operation’s spending. To obtain the re-sult with high quality in both stabilization and retargeting, we hope we can use recent outstanding stabilization and retargeting techniques, but no matter we do which oper-ation firstly, previous operation will cause that the latter operation can not proceed or fail; Can we integrate video stabilization and retargeting in one routine? In this paper, we present an integrated approach which can solve video stabilization and retargeting at one time. The central component of our approach is a least square minimization which warps video to the target resolution and achieves high quality camera motion for a wide range of videos. Retargeting problem was solved by employing motion in-formation and by distributing distortion in both spatial and temporal dimensions, and stabilization problem was solved by smoothing camera motion and frame homography. Experimental results confirm the effectiveness of the proposed approach.In the fifth chapter, we proposed a parallel texture synthesis method. PatchMatch is a randomized algorithm for quickly finding approximate nearest neighbor matches between image patches. Because local coherency and global similarity are considered, PatchMatch is an effective way to synthesis texture. But local coherency in PatchMatch propagation phase will introduce conflict between image patch blocks and can not be parallel executing, And random search phase of PatchMatch lost accuracy on sample patch searching. By stricting propagation distance in a small range, we achieve parallel execution ability. By replacing the random searching with kNN algorithm we overcome the problem of miss accurate sample patch. Our efficient texture synthesis algorithm implement on GPU, offers substantial performance improvements over the previous state of the art(20-100x), enabling its use in interactive texture synthesis and large texture synthesis.