Comfortable Research Of Stereo Image Based On Visual Attention Mechanism

In the process of acquisition,compression,storage,transmission and display,stereoscopic images are disturbed by the outside world,which will lead to image degradation and seriously affect people's viewing experience.Therefore,the assessment of stereoscopic image's visual comfort is one of the main problems that need to be solved in stereoscopic image technology.The human visual system has visual saliency and hierarchical characteristics when viewing images.Visual saliency refers to the degree of attention of the human eye when the image is viewed in different regions;the hierarchical characteristics refers to the process of abstraction and iteration of the human brain in the process of image processing.This paper mainly completes the work of two aspects:(1)This paper uses the visual attention mechanism for the first time to research how the brightness factor influences the visual comfort of stereoscopic images.(2)Based on the hierarchical characteristics of brain cognition,a convolutional neural network is proposed to study the visual comfort of stereo images.(1)In the first part,the saliency stereoscopic image is obtained by combining a disparity map and a 2D salient map(in this paper,the saliency stereoscopic image is verified by an experiment using eye tracker);then using the improved approximation method to reduce the subjective experimental data,through a large number of subjective experiments and experimental analysis obtain the comfortable stereoscopic image brightness matching map and difference map.The experimental results show that the maximum brightness of human comfort is 197.6,the lowest comfortable brightness is 35.7,the maximum brightness difference is 102.8.The brightness range proposed by this paper can better reflect the comfort of stereoscopic image,which provides a new basis for the comfort assessment of stereoscopic image and the improvement of stereoscopic display technology.(2)The convolution neural network is used to simulate the human perception to process the image.In this method,the left,right and disparity images of stereoscopic images are combined into an image as input of the network,and the weights of the net are obtained by training.Finally,the comfort value of the image blocks is weighted as the whole image comfort value by using the salient characteristics of human visual system.The model uses a lightweight network structure,which contains 3 convolutional layer and 3 full connection layer,through less network parameters achieving better assessment results.Meanwhile,ReLU is employed as activation function.The experimental results show that the proposed algorithm has superior performance in the LIVE 3D phase-I,LIVE 3D phase-II and MCL datasets,and the Pearson correlation coefficients can reach 0.953,0.942 and 0.943 respectively.The assessment model established by our method can well predict the comfort of stereoscopic images.