Research On Blind Stereoscopic Image Quality Assessment Algorithm Based On Binocular Perception Model

Objective stereoscopic image quality assessment(SIQA)algorithms are mainly used to evaluate the quality of stereoscopic scene displayed in many stereoscopic image processing applications.SIQA algorithms are useful for parameter setting and performance optimization of various image processing systems.Research on SIQA algorithms is very valuable for potential applications.The goal of SIQA algorithms is to automatically predict the perception quality of a stereoscopic image.At present,however,the complex working mechanism of human visual system has not been fully understood.It is of great challenge to design a prediction model that is consistent with human visual perception.Current researches on SIQA have explored some functions and characteristics in human visual system.However,there are still some challenges to be addressed,such as some binocular perception characteristics are not fully considered in current SIQA models,the prediction accuracy on evaluating asymmetrically distorted stereoscopic images is not high,and the computational cost of feature extraction is high.Based on the existing SIQA models,this thesis further investigates the perception characteristics of human binocular vision,and proposes some new blind binocular perception based SIQA models with the improved perception feature extraction and feature mapping methods.The main research work is as follows:(1)The SIQA methods without using disparity information usually have the problem of low prediction accuracy in evaluating asymmetrically distorted images,since these methods do not consider the binocular visual perception.To address this problem,we propose a blind SIQA algorithm using synthetic image of unmatched wavelet sub-bands.Based on the human visual property that some cells in human visual cortex are sensitive to visual input of different frequencies and orientations,the stereo pair are decomposed into multi-scale and multi-orientation sub-bands,and the wavelet sub-bands at the same scale and the same orientation obtained from the left and right views are fused into synthetic images without using disparity information.Then,we extract perceptual features that can better describe the quality changes of the stereoscopic image by using non-zero generalized Gaussian distribution model and Pearson linear correlation coefficient.The experimental results show that the proposed algorithm improves the prediction accuracy of asymmetrically distorted image while remains low computational cost.(2)Aiming to solve the problem that the reference stereoscopic images used in full-reference SIQA algorithms are difficult to obtain in practice,we propose a new no-reference SIQA algorithm based on binocular structural similarity and monocular feature fusion.Based on the fact that human eyes are sensitive to structure changes,we firstly generate the binocular structure similarity map between the left and right views to obtain statistics that can characterize the structure changes in binocular images.Meanwhile,monocular statistics are extracted from the color and gray-scale space to describe the structure changes in monocular images,and then monocular statistics extracted from the left and right views are fused into binocular fusion features by using monocular feature fusion coefficients,which are calculated by using the entropy and the standard deviation of monocular images.Finally,the binocular structure similarity features and the binocular fusion features are combined together for estimating the quality score of the stereoscopic image.In this method,we also employ the binocular structure similarity features and the difference features of monocular images to classify the symmetrically and asymmetrically distorted images.The experimental results demonstrate the effectiveness of the proposed method.(3)SIQA models based on fusion image usually use a single fusion image to simulate the convergence of the two eyes in human brain.However,there are ocular dominance columns in human visual cortex,and the behavior of binocular cells in the left and right ocular dominance columns cannot be simulated by using a single fusion image.To solve this problem,we propose a blind asymmetrically distorted SIQA algorithm based on ocular dominance theory.Firstly,gradient magnitude response of the left and right views is employed to model the visual inputs.Then,based on the ocular dominance in human visual system,two fusion images are generated by taking the left and the right images as dominant eye input,respectively,for perception feature extraction.Finally,a support vector regression model based AdaBoosting algorithm is employed to map the perception feature vector into the stereoscopic image quality score.In this method,we also compute the feature distances between binocular features and between monocular features to classify the symmetrically and asymmetrically distorted images.The experimental results show that,compared with the state-of-the-art SIQA algorithms,the proposed algorithm improves the prediction accuracy on asymmetrically distorted images.(4)Aiming to solve the problems that the sizes of stereoscopic image databases are limited to fully train the deep convolutional neural network(CNN)and the extracted features are not effective enough in current deep CNN-based SIQA methods,we propose a blind SIQA algorithm based on statistical feature extraction method and deep CNN.A three-column transfer feature extraction model is proposed to capture perception features from the stereo pair of original size,and a mixed image composed of the gray-scale stereo pair and the disparity map.Specifically,a deep CNN pre-trained on large-scale image classification database is firstly transferred to SIQA application.Then,the statistical feature extraction method and the global average pooling method are applied to reduce the output feature dimension of different layers in the transferred CNN.Finally,the features extracted from three transferred CNN are integrated together to form a feature vector of fixed length for stereoscopic image quality mapping.The experimental results indicate that the proposed method further improved the prediction accuracy on both symmetrically and asymmetrically distorted stereoscopic images.In conclusion,this thesis investigates the method of developing blind SIQA methods that are consistent with human visual characteristics based on human visual properties,including multi-channel property,sensitivity to structural changes,ocular dominance columns,and neural network.The proposed methods can obtain more descriptive perception quality features and improve the prediction accuracy of SIQA algorithm.