Blind Image Quality Assessment Based On Sparse Representation Fused With Transfer Learning

With the popularization of information technology such as Internet of things(IOT),cloud computing,big data,artificial intelligence,images are widely used as an important information carrier.However,the photographic environment,transmission and storage,and other factors will cause distortions in an image,and then affect the visual effect(image quality)of the image.Objective image quality assessment is to evaluate the quality of an image by simulating human visual perception using computer,among which blind image quality assessment refers to evaluating the image quality without considering information of original image and the type and quantity of distortion.The degree of proximity between blind assessment and subjective assessment is high,thus,blind assessment has better generality ability and scalability,higher research value and wider application prospect.In the major project of High-Resolution Earth Observation Systems,a research topic of making the consistency of subjective and objective assessment achieving more than 95%is proposed in Multi-Spectral Stereo Image Compression project.Aiming at improving the consistency of subjective and objective assessment of the evaluation method,this paper firstly proposes a variety of blind image quality assessment methods combining HVS characteristics,then proposes the evaluation method in conjunction with transfer learning,and finally provides the evaluation method which can be implemented on the embedded hardware platform.Specifically,it includes the following research content:Firstly,combining the visual attention and sparse representations of HVS,the sparse features of salient regions(SFOSR)method for blind image quality assessment is presented.This method has improved the CORNIA method based on the codebook in three parts.The first part is to use the salient information conditionally to filter the local feature descriptors of the image,replacing the random extraction of local feature descriptors,effectively eliminating the image redundancy information,increasing the expression ability of the dictionary to the local key information,and reducing the number of dictionary atoms in the evaluation effect.The second part is using sparse coding algorithm to train over-complete dictionary and calculate coding coefficient.Comparing with the coding coefficient of clustering analysis and principal component analysis,the representation coefficient of sparse coding is more suitable for quantifying image distortion degree,and conforms to the sparse representation characteristic of HVS.In the third part,the feature extraction model of Max-Pooling in conjunction with l1 norm is proposed,and the output features have strong robustness.Experimental results prove that the SFOSR evaluation method reduces the number of dictionary atoms and improves the subjective and objective consistency of the evaluation,and has good generalization ability.Secondly,the dual-scale sparse coefficient energy splitting features(DSSEDF)method for blind image quality assessment is proposed from the viewpoint of multichannel characteristic of HVS and coefficient matrix decomposition.For image set from cross down-sampled binary tree structure,two-scale sparse coefficients of the image are obtained by sparse coding,respectively.Comparing with the single-scale analysis of SFOSR method,double-scale analysis conforms to HVS multi-channel characteristic and improves the ability of information description.The feature extraction method of Max-Pooling in conjunction with l1 norm can cause information loss and waste.DSSESF adopts the singular value-based energy splitting method to decompose each expression coefficient matrix,then performs the feature extraction operation,and effectively reduces the loss of information.The experiment shows that the DSSESF method combining the multi-channel characteristic of HVS improves the prediction accuracy of the image quality,but the generalization ability is not obviously promoted,and the stability is decreased slightly.Thirdly,as SFOSR and DSSESF are dealing with small-samples of high dimensional space,the stability of predictive model trained from traditional machine learning methods gradually decrease with the increase of feature dimension,and the generalization ability is not significantly improved.Aiming at this problem,this chapter presents the method of sparse features in conjunction with transfer learning via sample instance for blind image quality assessment.Completing knowledge transfer requires building auxiliary data sets which consist of feature sets of auxiliary database(VOC-Distortion)and corresponding objective annotation sets(Predicted-Labels),auxiliary samples are generated through distortion processing,and Predicted-Labels are obtained through objective evaluation of auxiliary samples.VOC-Distortion contains a large number of samples,the transfer learning algorithm via sample instance TrAdaBoost.R2 can transfer useful knowledge from auxiliary data,and help the high dimensional feature to train predictive model.Transfer learning adopts the correlation between datasets to improve the fitting effect of predictive model,overcoming the limitation of traditional methods to deal with small-samples of high dimensional space,improving the prediction accuracy of model,and enhancing the stability and generalization ability.In addition,an assessment method based on feature self-learning transfer is proposed,which is to study the dictionary of spatial features from VOC-Distortion,and the dictionary adopts self-learning transfer method to obtain more robust reconstruction features of spatial domain(RFOSD),then using the traditional machine learning method to train predictive model.The experimental results show that transfer learning improves the validity and stability of small-sample learning in high dimensional space,and the assessment method of transfer learning achieves higher subjective and objective consistency,and has strong stability and generalization ability.Finally,aiming at the problems such as complex computation and difficulty of hardware realization,the assessment method of small-scale over-completed dictionary's projection features(SSODPF)is proposed.SSODPF calculates the projection features of the input image on a small-scale dictionary by using distance between its projection features and the standard projection features to quantify the image quality,effectively reducing the computation.The multi-spectral compression images taken by High-Resolution Earth Observation Systems are used as experimental objects,and the results show that the quality prediction score of the SSODPF method is monotonous with compression ratio,and the subjective and objective consistency satisfies the project requirement.On the multi-core embedded platform of TMS320C6678,the hardware parallel acceleration is accomplished by using the feature splitting method,this method reduces the computing burden,and the SSODPF method is finally realized in real-time assessment of multispectral compressed image quality.This paper opens out the research of blind image quality assessment based on the feature extraction and prediction model,and the proposed methods have good robustness and ability of real-time,among which the method in conjunction with transfer learning provides a new idea for the research of blind image quality assessment.