Research On Light Field Image Quality Evaluation Based On Feature Fusion

With the rapid development of information and digital media technology,people’s demand for visual content services has grown rapidly.As an emerging visual medium,Light Field(LF)can bring users a good sense of immersion and presence visually,and it has been widely used in the field of computer vision,such as 3D reconstruction,virtual reality and depth.estimate etc.However,in the processing of light field images at different stages,distortion effects will inevitably occur,resulting in the deterioration of image quality.In order to guide and supervise the acquisition,processing and application of light field images,it is crucial to design a light field image quality assessment model that is consistent with the Human Vision System(HVS).At present,many image quality evaluation methods have been proposed,but they are not applicable to light field images containing 4-dimensional structural information.The traditional image quality evaluation model is insufficient for the feature extraction of light field images.Existing light field image quality evaluation algorithms often rely on feature extraction with high complexity,and most of them only perform quality evaluation on a single form of two-dimensional image,such as light field.Sub-Aperture Images(SAI)in the spatial domain or Epipolar-Plane Images(EPI)in the angular domain of the light field.In this paper,the deep learning method is used to evaluate the quality of light field images from the perspective of light field space domain and angle domain,and then extended to the light field image quality evaluation based on feature fusion.The main work of this paper is as follows:(1)A light field image quality evaluation method based on spatial domain is proposed.Due to the SAI under the same distortion type,there are only slight differences between adjacent images and most of the image contents are similar.In this paper,a light field adaptive image quality assessment network model(LFSA IQA Net)is designed.The light field spatial domain SAI is divided into image stacks in four directions according to different viewpoint angles,and the LFSA IQA model adaptively generates the parameters required by the quality prediction network according to the different input image stacks.The experimental results show that the proposed method can well conform to the subjective perception characteristics of human vision for light field images.(2)A light field image quality evaluation method based on angle domain is proposed.The EPI reflects the angular consistency of the light field,so the angular distortion of the light field image can be evaluated using the feature information in the EPI.Since EPI has obvious distortion effect only in the edge region of the object,in order to reduce the computational complexity and improve the training speed,this paper uses the low-complexity EPI edge-aware region block to replace the entire EPI as the input of the network model.Quantitative analysis of the experimental results shows that the proposed method can accurately reflect the subjective judgment of distorted light field by human visual characteristics.(3)A light field image quality evaluation method based on feature fusion is proposed.The multi-dimensional features of the light field can be represented by the SAI in the spatial domain and the EPI in the angle domain.In this thesis,an image quality assessment network model(LFFF IQA Net)for light field feature fusion is designed.Global semantic features and multi-scale content features are fused in the channel dimension,and the obtained multi-scale content fusion features and global semantic fusion features are involved in the generation of dynamic parameters and the prediction of quality scores,respectively.The qualitative and quantitative results of the experimental results show that the proposed method exhibits the excellent performance on light field image quality assessment compared with existing quality assessment algorithms.