Research On Content-aware Image And Video Retargeting Methods

The diversity of multimedia display device and a wide range of size change make the source information like image or video retargeting problem become a research hotspot. Content-aware retargeting method can not only scale source information to the target size, but it keeps the important content and the whole structure of the source information as much as possible at the same time.Under the support of National Natural Science Foundation Project "Sparse reconstruction method for optical molecular tomography combing a three-dimension statistical deformable model"(Granted No.61372046) and Shaanxi Natural Science Foundation Research Project "The research on content-aware image retargeting method"(Granted No:2014JM8346), this thesis researches and analyzes the research status of the problems about images and video retargeting in depth. On this basis, the 2D images and 3D video content-aware retargeting problems are studied by employing the depth information, discrete, continuous and hybrid processing methods etc. The main work completed in the thesis is listed as follows:(1) The importance map of the color-and-depth information and its extraction algorithm is proposed. The depth information is from the RGB-D camera and reflects the distance information of objects in the scene. The depth map can be got by researching the original depth information with the help of corrosion morphology, inversion, Mean-shift process and Gaussian weighted operation. The joint depth map, saliency map, gradient map can be used to extract more accurate importance map which can guide retargeting algorithm to make the algorithm performance be improved greatly. At the same time, in order to further prove the validity of depth information, the combined algorithm is applied to stereo images, which also got similar results.(2) An image retargeting algorithm based on Improved Seam Carving is proposed. Making use of the directionality of gradient improves the energy map of the original seam carving algorithm; The application of threshold technology on energy map obtains the binary map to check the pixels on the optimum seam and mark the undeletable points based on the binary map; Adjacent seams can be merged by using low-pass filter to realize image downsampling; By enabling stop mechanism, the seam carving operation can be stopped in time in order to turn to other retargeting methods before the new seam lead to the visual distortion. The experimental results show that the ISC algorithm not only has good resizing effects, but overcome the shortcomings of all kinds of seam carving algorithms, such as the distortion, the excessive deletion etc.(3) The image retargeting algorithm based on Orthogonal Movement of Gridlines is proposed. Firstly, it uses the improved Achanta’s method to extract the main object of the source image, the joint gradient map and the saliency map to identify the important areas of the source image; Secondly, by calculating the optimal grid line displacement, OMG not only keeps the size of important areas but also protects the aspect ratio of main object; What’s more, the lower and upper thresholds were used to restrain the distortion caused by excessively narrowing and widening grids; Finally, in order to make important areas occupy a larger proportion in the output, an edge discarding process was introduced to assign wider space to the important area for reducing the distortion. The experimental results show that the scaling results of the OMG algorithm not only have less distortion but also are obviously better than the results of the contrast algorithm in the aspect of retaining the important areas and the main objects in the image.(4) The video retargeting algorithm based on Quick Seam Carving is proposed. After the video analysis of the source video, the camera motion parameters between successive frames can be estimated. According to the motion parameters, we can build the background image and segment prospect object; Copy prospect object pixel to the background image in order to form the frame assembled figure; By the application of Expended Seam Carving algorithm, the optimal pixels line is calculated and identify the robust seams through relevant standards; Using the camera motion inverse parameters can map the robust seams inversely into each frame for further processing which can realize video retargeting. The experimental results show that the QSC algorithm can resize videos well and greatly reduce the complexity of the previous video retargeting strategies in time and space.(5) An image retargeting algorithm based on the Expended Seam Carving is proposed. Using the edge detection and Hough transformation, the straight line maps are extracted from the source image; The original seam carving algorithm tests whether the optimal seam which are produced in each iteration exist the intersection point with the straight line. If no intersection point, then get into next iteration. If there is any intersection, then increase the energy value in the intersection point of the compensatory energy diagram to update and compensate for the energy diagram; The compensatory energy diagram can be attached to the map of important degree in order to further update the map of important degree, thus reducing the possibility of the seams which are produced in the next iteration and this iteration passing through the straight line in the way of adjacent intersection points. Experimental results showed that ESC algorithm can not only produce sufficient seams, but makes the retargeting distortion be in the acceptable range, which successfully promote the image seam carving algorithm which is applied to video processing.