Research On 3D Visual Media Retargeting Methods

Three dimensional visual media(image and video)is becoming more and more popular because of its shocking three-dimensional effect and the very powerful visual sensory experience.There are various display terminals with different resolution and aspect ratio nowdays.When the resolution of the image and video does not match with the display screen,the problem of how to match the display is called the retargeting problem.Content-aware retargeting based on the user's viewing experience is widely concerned.This dissertation is devoted to the research on 3D visual media retargeting methods.By analyzing the perceptual characteristics of human visual system,we research on 3D visual media retargeting from three factors: visual attention selection,visual comfort optimization and visual depth perception enhancement.The main contents include the following four parts:(1)Taking into account the existing stereo image retargeting methods do not fully consider the stereo visual attention factors,this paper proposes a stereo image retargeting method based on stereo visual attention model.The method of seam carving for single image is extended to binocular stereo images by ensuring the geometric consistency of the corresponding pixel of the left and right views.Further,according to the perception characteristics of human visual system,we construct the objective quantitative model to describe visual attention for stereo image considering 2D spatial color feature and depth perception.This model is used to simulate the degree of interest of the human eye to different contents and regions of the scene,and will be added to the stereo image retargeting framework guiding the algorithm to protect the important objects in the scene and improve the experience effect.(2)We research on visual attention guided video retargeting method based on seam carving.As for video retargeting,each frame independently processing will bring the jitter.Therefore,it is a very important task to ensure the time domain correlation between adjacent frames.Our method first constructs a visual attention model based on depth information,gradient,saliency and motion detection;then we construct intra frame energy and inter frame energy.Combining with the visual attention energy terms,the total energy is constructed to seam carving for video.(3)A stereo image retargeting algorithm is proposed to enhance quality of visual experience.At the time of retargeting,visual comfort and depth perception of the scene are optimized by adjusting the perceived depth.Using the framework of warping and according to the viewing environment parameters,we construct linear depth mapping energy,depth sense enhancement energy and depth structure preservation energy.Further,combining with the related energy terms of shape preserving,we get the optimized total energy function.So as to enhance the visual comfort and depth sense,the depth distribution of the scene is optimized to adjust binocular disparity of stereo image at the time of retargeting.(4)A multi view video system retargeting scheme for virtual viewpoint rendering is proposed.First,the depth of scene is converted into binocular disparity in the display space according to camera parameters and baseline distance.Further,the disparity will be converted into perceived depth of 3D space according to the principle of the stereo vision.Through the combination of depth perception as the third dimensional coordinates,we construct total energy function from three aspects: shape preservation,line maintenance and visual comfort optimization.Based on collaborative optimization of mesh deformation and depth adjustment,color image can be resized to the target resolution.And depth image is resized in accordance with the color map.Then,by fitting the value before and after optimization of third dimensional coordinate(depth perception),we map the original value of depth image to a new depth value.Finally,depth image-based rendering(DIBR)is used to obtain the retargeting image of the virtual viewpoint.