Data Driven Image Perception And Quality Assessment

With the rapid development of digital media technology,human beings have entered an information era,and massive images play an important role in daily lives.Since noise exists inevitably in every aspect such as image acquisition,transmission,storage,and display,there is quality degradation in images,which is called image distortion.In the streaming application,the users' demand for high quality experience is increasing.Therefore,it is necessary to evaluate the image distortion.In order to ensure a good viewing experience for users,image perception and quality assessment has become an indispensable task.With large-scale image data,it is very helpful to discover knowledge of data,which is data driven image quality assessment method.In this dissertation,we aimed to discuss image perception and quality assessment by using data driven algorithms.The main contributions of our work are as follows:1.We propose a no reference image quality assessment method based on multi-expert CNN.In real applications,there are various distortion types,a single existing image quality assessment method cannot perform the best for every distortion type.Firstly,the distortion type is identified.Secondly,an expert network is trained for a specific distortion type.Finally,the outcomes of recognition network and expert networks are fused.Experiments show a great improvement in the expert network compared with the single CNN trained on all data,which indicates that the features learned by expert network are more discriminative.The experimental results on the LIVE II and CSIQ datasets show that the prediction results of the proposed method based on have a strong correlation with the human visual system.2.We develop a deep learning based model to predict the Satisfied User Ratio(SUR)for compressed images.According to the Just Noticeable Difference(JND)characteristics of the human visual system,people cannot perceive the distortion until it reaches a certain threshold,which is called JND.Traditional JND-based models mainly focus on pixel domain or sub-band domain,but distortions caused by one pixel or few pixels that out of the range of their threshold may not be perceived because of visual masking effects.In this thesis,we focus on Picture-level JND(PJND).Individual's PJND varies due to their physiological structure,situation of vision health,and knowledge background.SUR is the ratio of the subjects who are satisfied with the quality in the group.A convolutional neural network based model is proposed to predict the SUR for compressed images.Given a certain SUR,we can predict the maximum coding parameter,the maximum distortion level,and the minimum bit rate.The research results can be directly applied to save bit rate in streaming.3.We construct PJND based symmetric and asymmetric stereoscopic image datasets.Stereoscopic images provide an immersive visual experience for viewers,which can better reflect the real perception of viewing the real world.We explore the PJND of stereoscopic images by subjective tests,including test environments setting,stereoscopic images collection,and the results post-processing.We find that SUR decreases with the distortion level increases,which indicates that distortion degrades users' satisfaction.In asymmetric compression,there is a difference threshold of the left and right views of the PJND,the mean value is 2.12 d B for JPEG2000 compression and 2.38 d B for H.265 intra coding.We construct a symmetrically compressed stereoscopic image dataset named SIAT-JSSI and an asymmetrically compressed stereoscopic image dataset named SIAT-JASI,and make them accessible to the public.4.Considering the binocular rivary and binocular fusion mechanisms of the binocular vision,we propose a novel method to predict the SUR for compressed stereoscopic images.Firstly,we extract features from reference stereoscopic images and distorted images,respectively.The features include stereoscopic image quality,monocular visual features,and binocular visual features.Then we select key features from those features,concatenate them together,and feed them into a Support Vector Regression(SVR)to learn a mapping function from feature space to SUR value.Transfer learning is used to reduce overfitting because of the small dataset.Experimental results on SIAT-JSSI show a high performance.