Research On Perceptual And Statistical Model For Quality Assessment Of Stereoscopic Images

With the progress of information technology,imaging acquisition technology and communication technology,aiming to get vivid and immersive experience,2D images and videos are transformed into stereoscopic images and videos,becoming the research hotspot in the field of multimedia signal processing.Stereoscopic image technology is widely applied in modern life.People can create three-dimensional(3D)advertisements with new,unusual and special features by using 3D imaging processing technology.Meanwhile,3D films and 3D games with a strong sense of visual impact and immersion have become the focus of the entertainment industry.In addition,the explosive growth of 3D contents has also greatly promoted the development of 3D display and projecting equipment.However,confined by the available 3D technic(i.e.,acquisition,compression,storage,transmission,reconstruct),a variety of image distortions can be inevitably introduced in an image processing system.The distortions of stereoscopic images will seriously affect people's quality of experience(Qo E),and even cause some visual health problems.Therefore,stereoscopic image quality assessment(SIQA)plays a vital role in the application of 3D image and people's Qo E.Exploring how to establish an objective SIQA metric for predicting the quality of 3D image that is consistent with subjective opinions.It is considered as a research hotspot in the field of 3D image processing.Moreover,SIQA is great theoretical and practical significance to promote the 3D application and popularization.Since the complexity of human visual system(HVS),SIQA has evolved into an energetic field with the intersection of neural visual physiology,visual psychology,machine vision,multimedia signal processing.Further,since the HVS is the ultimate receiver of 3D image,how to accurately simulate the mechanism of the HVS,which plays an important role in developing SIQA metric.At present,the existing SIQA algorithms still have many problems,such as human visual characteristics considered inadequately,the low performance,the high computational complexity and the limits of generalization.To solve these problems,this thesis aims at constructing some objective SIQA models which is more consistent with the subjective opinions and proposes some effective and useful SIQA metrics.The major contributions and innovations of this thesis are as follows:1)Two full-reference(FR)SIQA models are proposed.(1)Earlier SIQA metrics straightforward applied 2D image quality assessment(IQA)metrics to the left and right views of 3D images,respectively,and then aggregated the two quality scores to yield the final perceptual quality of 3D images.Obviously,these metrics are not in line with the characteristics of the HVS,and the experimental results also show that this kind of methods do not achieve satisfactory performance.In this thesis,according to binocular fusion and rivalry of the HVS,the 2D IQA methods and binocular energy-contrast perception are jointly considered for SIQA.The weights of binocular energy-contrast perception are computed in order to construct a dual-channel SIQA model.Experimental results show that the proposed method achieve high consistent with subjective scores.(2)According to the neurophysiological findings,the HVS is a distributed information processing system,and the visual cognition is divided into monocular perception and binocular perception.Based on these theories,this thesis presents a FR SIQA metric by simulating binocular visual properties.The proposed scheme first simulates the receptive fields of simple cells and complex cells.Subsequently,various quality-aware features are extracted from the responses of area V1.Finally,kernel ridge regression(KRR)is used to simulate a nonlinear relationship between the quality-aware features and objective quality scores.Experimental results show that the predicted objective quality scores are consistent with the subjective MOS scores.2)The FR SIQA model usually requires the reference information of stereoscopic image.However,in most practical application,the reference information is rarely accessible completely,which severely limits the application scope of those FR SIQA models.To solve this problem,this thesis presents a reduce-reference(RR)SIQA using natural scene statistics(NSS)and structural degradation which combines NSS-based features with HVS-based features.Experimental results confirm that the proposed model has the advantages of excellent performance,few referernce information,low complexity and strong portability,etc.It also solves the problem of SIQA metric which relies on the full information of reference image.3)The RR SIQA model still need to extract some features from reference images,which is still an obstacle in some particular applications.On the other hand,RR SIQA metrics need to transmit or embed the extracted features with the distorted 3D image in complex communication networks.This will introduce an additional burden for 3D image processing system.In response to these challenges,this thesis proposes a blind SIQA model without the information of reference image using jointly statistics of binocular rivalry and orientation tuned responses.Subsequently,all quality-aware features extracted from the distorted 3D image are mapped to subjective quality scores via using support vector regression(SVR).Experimental results demonstrate that the proposed metric achieve good performance and robustness,and also solves the problem of SIQA metric without the reference information of 3D image.In conclusion,this thesis takes in-deep researches on issues of SIQA,such as FR SIQA,RR SIQA,NR/blind SIQA models,proposes the corresponding technical solutions and methods for SIQA design.A large number of experiments are performed on the public 3D databases,and the results show that the proposed schemes achieve good performance,and superior to the current existing methods.