Research On Visual Optimization And Evaluation For Terminal Display

As the demand for multimedia applications grows,more and more display devices appear on the market,such as stereoscopic 3D display devices(3DTV)and head-mounted display devices(HMD).Because the generated content in the display process is separated from the terminal device,adapting the display content in a specific format is an important research.In addition,content quality degradation and distortion will occur during content capture,encoding,and transmission,and the visual quality evaluation based on the user's viewing experience has also received widespread attention.This paper is aimed at 3DTV and HMD display devices,and proposes stereoscopic 3D image display optimization methods and panoramic image quality evaluation methods.The main contents are as follows:(1)For addressing the problems of depth enhancement and resizing of stereoscopic images,this paper presents a depth-adjustable stereoscopic image retargeting method using pixel fusion technique.The traditional methods enforced depth-preserving constraint on the left and right images resulting in poor depth adaption.To solve this issue,in the latest Seam Carving-based pixel fusion technology,the depth range is adjusted by changing the widths of unmatched region to meet the viewing comfort,and the object scaling energy and parallax consistency energy are converted to calculate the optimal scaling factor via the least squares.Meanwhile,the left and right seams are aligned to reduce the effect of image distortion on viewing.(2)To enhance the compositon of stereoscopic images,we present a Delaunay-preserving warping model for stereoscopic image recompostion.Firstly,objects,feature lines and other areas that need to be adjusted for reconstructed stereoscopic images are selected,so Delaunay grids are established.Then,energy constraints from image quality energy,aesthetics energy and depth adaptation energy are involved in the framework to solve the optimization for image aesthetic enhancement.Different from the existing stereoscopic image editing methods,the main innovation of our method is to achieve a tradeoff in balancing information loss and reducing semantic distortion for recomposition and retargeting applications,while improving parallax adjustment geometric model for stereo object scaling.(3)Considering the research in thumbnail generation for stereoscopic images is still relatively lacking,this paper presents a novel thumbnail generation technology for stereoscopic images based on multioperator.Firstly,the warping technique is used to retarget a stereopair into six-scale resolutions with different contexts,and the disparity is uniformly adjusted to a certain value based on just noticeable depth difference(JNDD)model,which overcomes the issues that 3D perception in stereoscopic thumbnail is uncontrollable and the sense of depth disappears in low-resolution stereoscopic images.Then,the six-scale images are cropped via cropping network,and are optimized to a target resolution based on the designed image visual representation energy.(4)Considering the high requirements for omnidirectional video compression,we propose an objective quality evaluation method to assess quality loss in encoding omnidirectional videos.The image content and motion information of 360° video are described by the signal characteristics of the spatial and frequency domains.From the perspective of multiple projection planes and time grouping,the multi-channel information aggregation score is further considered to effectively measure the video quality of 360°.(5)Compared with the developing full-reference models,researches on no-reference quality evaluation of panoramic video are still limited.With the above motivation,we propose a blind quality evaluation framework for panoramic videos.On the one hand,from the perspective of the viewport format,we eliminate the sampling distortion of panoramic videos in equirectangular projection(ERP)format via cubemap projection(CMP)format projection.On the other hand,we use a spatio-temporal convolutional network to fully extract the intra-frame and inter-frame information for more comprehensive modeling of panoramic video features to predict quality scores.