Research On Image And Video Retargeting

The proliferation of computers, HDTVs, mobile phones, PDAs and other digital devices enriches people's daily life significantly. At the same time, it brought a new research topic to computer graphics and computer vision. That is image and video retargeting which aims to adapt an image to heterogeneous devices with different sizes and aspect ratios other than originally intended. The existing commercial retargeting solutions are still very limited. The results from these methods often suffer from objectionable artifacts, such as stretching. With the rapid development of digital displays, image and video retargeting becomes a hot topic in the area of computer graphics, vision, multimedia, and human computer interaction. Retargeting is a special case of image and video warping problems. Particularly, the goal of retargeting is to change the size and aspect ratio of an input image and video to fit the target display. The change of aspect ratio will necessarily introduce the geometry distortion to the retargeting result. Accordingly, the research on image and video retargeting research mainly focuses on how to avoid introducing visual distortions to results. While there has been a rich literature of retargeting research, the results from existing methods often suffer from distortion still. Meanwhile, existing work mainly focus on developing new retarget mechanism, the study of how people perceive retargeting results is unfortunately neglected.This thesis first surveys the existing work on image and video retargeting. Based on this survey, this thesis presents a novel image retargeting method via non-homogenous image warping. Then, this thesis describes an efficient and effective video retargeting method based on the novel idea of warp propagation. Finally, this thesis introduces user studies that explore people's perception of a special retargeting result, image stretching (homogeneous resizing), and provide insights for better retarget result evaluation and retargeting method design. The contributions of the thesis mainly include:1. Image retargeting methods often destroy the global image structure and lead to obvious visual distortions. In this thesis, we propose a novel image retargeting method via non-homogeneous warping. This method introduces a patch-linking scheme that can better preserve the global image structure. The method uses different strategies for upsizing and downsizing, the results accord with the target display. Experiments with a variety category of images demonstrate the effectiveness of the method:2. Achieving both temporal coherence in the background and shape preservation for the moving objects simultaneously is challenging. In this thesis, we present a novel video retargeting approach that achieves these goals by designing a novel warp propagation scheme. Specifically, we introduce a motion history map that integrates motion information from multiple frames and propagates them across frames. This motion history map allows for graceful tradeoffs between temporal coherence in the background and shape preservation for the moving objects. Based on this motion history map, our method warps each video frame sequentially and achieves real-time performance. Experiments with a variety of videos demonstrate that our approach can robustly process videos with significant camera and/or object motion, and efficiently produce high-quality video retargeting results.3. Retargeting results evaluation is an important problem, but it is neglected so far. Retargeting result evaluation is subjective and complex, so it's difficult to design computational methods. We design a novel image retargeting result evaluation method based on studies on people's perception to the retargeting results. Specifically, our exploratory research designs and conducts a series of user studies to investigate people's perception of a special type of retargeting results:image stretching (homogeneous resizing) results. To our best knowledge, this is the first study that is formally designed to study people's perception on retargeting results. This study not only provides a guideline for formal retargeting result evaluation, but also provides ramifications for the design of image retargeting methods.