Research On 2D And 3D Image Retargeting Quality Assessment

Image retargeting is an image editing method,which aims to adapt source images to target display devices with different sizes and aspect ratios.With the development of display technics and the diversification of terminal devices resolutions,image retargeting has been received much attention in recent years.However,different retargeting methods have different retargeting attributes.For some retargeting methods,they have satisfactory performance for one kind of images,but poor results for others.Considering the problem of poor generality and applicability for existing 2D and 3D image retargeting methods,this paper focuses on designing evaluation methods for 2D and 3D image retargeting.The main contents of this paper include:(1)We propose a bi-directional similarity transformation measurement for 2D image retargeting quality assessment.In the method,similarity transformation matrix is used to establish the bi-directional transformation relationship between the original image and the retargeted image.The visual quality of retargeted image is evaluated via the bias of similarity transformation matrix and the area loss of local grid.Besides,we propose an improved quality evaluation method named TRASIM by combining local distortions and global distortions.Furthermore,a TRASIM-guided multi-operator image retargeting algorithm is proposed and achieving satisfactory retargeting performance.Experimental results on Retarget Me and CUHK databases demonstrate that the proposed quality assessment methods can accurately predict the quality of retargeted images.(2)We propose a 2D image retargeting quality assessment method based on hand-crafted and deep-learned features.First,the similarity transformation matrix is used to compute the local multi-scale structure distortion,and the local information loss is obtained according to the area loss of grids.Then,a deep learning network is employed as the encoder to calculate the texture and semantic features.Finally,the hand-crafted and deep learned features are fused to obtain the retargeted image quality by support vector regression.The evaluation result of the proposed approach is broadly in line with the subjective perception.(3)We propose a subjective and objective measurement for 3D retargeted images.In subjective evaluation,we build a 3D image retargeting quality assessment database named NBU-SIRQA.Subjective score of 3D retargeted image is obtained by subjective test and analyzed qualitatively.In objective assessment,we propose an objective quality evaluation method for 3D image retargeting base on depth perception,visual comfort and left/right image quality.The proposed database can be used for 3D image retargeting quality assessment,and the presented objective evaluation method can accurately predict the quality of 3D retargeted images.(4)We propose a transformation aware quality evaluation method for 3D image retargeting.The main idea of the method is to decompose complex 3D image retargeting algorithm into two individual processes: monocular image retargeting transformation and viewpoint transformation.In monocular image retargeting transformation,grid deformation and content loss features calculated based on pixel shifting are applied to evaluate the geometric distortion and information loss.In viewpoint transformation,we use grid deformation and content loss computed by disparity map to measure visual comfort and depth perception.Finally,individual quality components are fused to obtain the quality of 3D retargeted image.The proposed model is evaluated on a benchmark database,and shown to deliver highly competitive performance.