Research On Stereo Image Quality Evaluation Method Based On Monocular And Binocular Visual Characteristics And Deep Learning

With the development of society and the advancement of technology,more and more3 D images appear in our vision.However,these stereoscopic images can be distroted during collection,transmission,processing,storage,and compression,which can cause distortions on the display side and hinder people from understanding the objective world.In the field of stereoscopic image quality assessment,how to assess stereoscopic image quality objectively and effectively has become a challenging research topic.In recent years,the research of Human Visual System(HVS)has made major breakthroughs in the domain of biology,which can also promoted the research of stereoscopic image quality evaluation models based on monocular and binocular vision.Based on existing research works,two new stereoscopic image quality assessment algorithms based on monocular and binocular vision characteristics are proposed,which can be summarized as follows::1.a full-reference stereoscopic image quality assessment method based on monocular and binocular visual features is proposed.Especially,a region segmentation method based on disparity and Euclidean distance is developed to divide distorted images and reference images into Occluded region(OC)and a Non-occluded(NOC)region.Then JND is used to simulate monocular vision in OC area and BJND model is adopted to simulate binocular vision in NOC area.A Pooling Strategy of Global Edges(PSGE)is further applied to obtain a quality score of monocular and binocular features.In addition,for distorted and reference images,the features of local phase and local amplitude are also extracted and fused into a quality score of local features.Finally,both quality scores are pooled into the final predicted quality score.Experimental results on LIVE 3D database show that the evaluation results of proposed method are highly consistent with subjective scores,which can also demonstrate that the algorithm can effectively simulate the human visual system for stereoscopic image quality evaluation;2.A no reference stereoscopic image quality evaluation network based on binocular vision characteristics(Stereo IF-Net)is proposed.Especially,four cross convolution based Binocular Interaction Modules(BIM)are proposed to simulate the interaction process between left and right eyes.And a Binocular Fusion Module(BFM)is also proposed to to simulate the binocular fusion of HVS.For the network,two convolution layers are used to extract the low-level visual features of HVS.Then four binocular interaction modules are adopted to simulate binocular interaction process in V2-V5 area.In higher level areas than V2-V5,a binocular fusion module is employed to simulate visual fusion process in the brain.And two dense layers are used to obtain local quality assessment scores.Finally,these local quality scores are fused into the overall predicted score of stereoscopic images.Experimental results show that the proposed binocular interaction module and binocular fusion module can effectively simulate the different binocular features of the human visual system,and Stereo IF-Net has a high consistency with subjective evaluation results.