No-reference Stereoscopic Image Quality Assessment Based On Binocular Visual Perception

Recently,with the development of stereoscopic imaging technology and multimedia applications,more and more stereoscopic images appear in our daily life,including 3D movies,three-dimensional games and so on.The research on stereoscopic image processing has also attracted the attention of both academia and industry.In the stereoscopic image processing,transmission and reconstruction,stereoscopic images are inevitably affected by noise which may result in the loss of visual information,and image quality degradation.Image quality degradation not only affects the user's visual experiences but also its own value.Therefore,stereoscopic image quality assessment is crucial for various stereoscopic image processing applications.Stereoscopic image quality assessment includes subjective evaluation and objective evaluation.It is well known that subjective evaluation is time-consuming and power consumption,and subjective evaluation results are influenced by environment and other factors.Meanwhile,subjective evaluation cannot be embedded in multimedia applications.Thus,it is urgent to develop objective quality assessment methods which can predict the visual quality of stereoscopic images automatically.Based on the exploration of perceptual characteristics of the human visual system(HVS),we will study the factors which have influence on visual quality of stereoscopic images.With these,we construct objective quality assessment models for stereoscopic images.The main contents of this paper are as follows.(1)Motivated by important visual perception properties of the HVS named binocular integration and binocular rivalry,we propose a novel no-reference(NR)quality predication method for stereoscopic images based on monocular and binocular features.When binocular fusion occurs,left and right views will be fused into a map,which means left and right views can be perceived by human eyes at the same time.Thus,we calculate the disparity map based on left and right views and extract disparity features,and integrate the disparity map with left view and right view to extract structure features.When binocular rivalry occurs,only left view or right view can be perceived by human eyes at a certain time,and they are perceived alternately.Specifically,only left view or right view can be perceived when binocular suppression occurs.And some studies have shown that luminance features can be used to perceive the image quality variation.Thus,we extract the luminance features of left view and right view.Finally,we use support vector regression to build stereoscopic image quality assessment model.And experimental results show that the proposed model can obtain consistent predicting results with subjective ratings.(2)The inherent inference mechanism in the brain demonstrates that the brain first analyzes the perceptual information and then extract effective visual information.Meanwhile,in order to simulate the inner interaction process of the HVS when perceiving the visual quality of stereoscopic images,we construct a two-channel convolutional neural network based stereoscopic image quality assessment method in this paper.First,we design a deep convolutional neural network to extract high-level features of left and right views for simulating the process of information extraction in the brain.Second,to imitate the intrinsic inference mechanism in the brain,we combine the high-level features of left and right views by convolutional operations.Finally,the information after interactive processing is used to predict the visual quality of stereoscopic image.Experimental results show that the proposed deep convolutional neural network based method can estimate the visual quality of stereoscopic image accurately,which also demonstrates the effectiveness of the proposed two-channel convolutional neural network in simulating the perception mechanism in the HVS.