Stereoscopic Image Quality Assessment Via Visual Perception And Deep Learning

Recently,three-dimensional(3D)media applications,such as virtual reality(VR)and 3D television(3D-TV),have been widely used in human daily life,which has driven consumers' interests in high-quality multimedia contents of 3D images and videos.Unfortunately,there exists a bottleneck of image degradation during the image transmission,driving the generation and development of the research on Stereoscopic Image Quality Assessment(SIQA)over the past few years.Based on the perception characteristics of Human Visual System(HVS)and the deep learning technology,this paper conducts in-depth research and analyses on SIQA tasks,and mainly proposes:1.A full-reference SIQA method based on the key characteristics of stereo vision.By analyzing the binocular characteristics of stereo vision in HVS,a full reference SIQA method based on the key characteristics of stereo vision was proposed.Firstly,the mathematical models for simulating binocular fusion and binocular rivalry effects were established,and multi-scale energy response maps and brightness maps were extracted from the two established binocular models.The corresponding feature similarity maps were generated by referring to the similarity measurement between the reference and distorted stereoscopic images.Finally,Local Binary Pattern was utilized to reduce the feature dimension,and Support Vector Regression technology was used to integrate and regress the feature vectors into the objective quality score.Experimental results showed that the proposed SIQA method is highly consistent with human subjective perception.2.a no-reference SIQA method by learning deep features of saliency regions.In order to overcome the dependence of SIQA methods on reference images,this paper also proposed a no-reference SIQA method by learning deep features of saliency regions.By applying the technologies of deep learning,the convolution neural network(CNN)models can adaptively learn the deep features related to the image quality.According to different network's inputs,two CNN-based SIQA models were proposed.Both of them adopt a patch-wise strategy and a saliency-guided local feature extraction method.Through the training procedure of local-to-global quality-aware feature aggregation,a final objective score can be obtained from the designed CNN model.The results of a series of experiments demonstrate the superior performance of the proposed CNN models compared to most state-of-the-art full-reference and no-reference SIQA methods by significant margins.In particular,compared to the one-column CNN model,the three-column CNN model can achieve better performance on most distorted image quality prediction.