Research On Content-aware Image/Video Retargeting

Development and upgrading of the display devices have been changing human life.People can browse digital media on devices anywhere and at any time. However,there are many kinds of display devices with different size and aspect ratio. If the sizeof image/video resources is not consistent with specification of display devices, It willinflunce the user’s experience and cause waste of screen space.The objective of image/video retargeting is to adapt an image to heterogeneousdevices with different size and aspect ratios without changing the primary imagecontent. In this thesis, several image retargeting methods have been proposed and asimple and fast video retargeting framework has been proposed. the details is asfollows.1. We proposed an improved seam carving method based on gray energytransferring. We analyzed why the artifacts are caused by by seam carving, and weproposed an energy transfer based method. The energy of removed seams istransferred to the pixels in its neighborhood. It increased the energy of these pixelsand decreased the probability by which these pixels are chosen as seams. Theproposed approach reduced the probability of neighboring seams and greatly reducedthe degradation of image quality brought by seam carving.2. We proposed a fast seam carving approach with strip partition and neighboringprobability constraints. Inspired by the non-homogenuous warping, we divided theimage into several strips. And for each strip, the objective size is estimated from itsimportance. For each strip, the fast seam carving scheme is utilized to resize it to theobjective size. This approach introduced the global constraints and resulted inreasonable suitable seam distribution in the whole image. it is can get a goodretargeting results quickly.However, the fast seam carving has not considered thedistribution of seams in a strip.Furthermore, We proposed another fast seam carvingapproach with strip partition and neighboring probability constraints. The neighboringprobability is obtained to describe the neighboring relationship between the seams in astrip. By combining the neighboring probability and their accumulated energy, theleast important seams are removed. The neighboring probability constraint ensuresthat the removed seams distributed non-neighboringly in a strip. It avoided abruptchanges in the scene and lead to an improved quality in the resized image. 3. We proposed a novel image resizing approach to combining improved seamcarving and warping. The above improved seam carving method could improve theimage quality to certain extent. In another words, after the same number of seams havebeen removed, the improved approaches could get better resized image than thetraditional approaches. However, after a lot of seams were removed, the improvedapproaches still result the loss of important contents in an image, and brings obviousimage degradation. For this problem, we proposed an approach of combining seamcarving and warping. Firstly, the seam carving was used to resize the image to a size.The content-aware image quality metrics (CAM: Content Aware Metric) wasproposed to evaluate the quality of resized images generated by seam carving. If theCAM is larger than the assigned threshold, the seam carving is stopped and thecurrent image is adapted to the target size using warping.4. We proposed a simple and fast video retargeting framework. Almost all of thecurrent video retargeting methods have two drawbacks. The first one is that thealgorithms are not practical. In order to get video of different target sizes, theapproaches should be repeated. Another disadvantage is that these approaches are notreal-time because of the heavy computation. We proposed a framework, whichincludes the peprocessing module and the real-time processing module. The mainwork of peprocessing module is to split the image into strips, to align the strips by theinter-frame correlation and to calculate the overlapped importance of each frame.After the preprocessing, a config file is generated and stored for the followingprocessing. In the real-time processing module, the video is resized to the target sizein real time by the information from config file. The different size of video could beobtained by repeating the real-time processing algorithm.