Stereoscopic Image Quality Assessment Based On Deep Learning

In recent years,3D stereoscopic films have been widely popularized.In order to enhance the comfort of watching 3D movies,many comfort evaluation methods have been proposed.All these methods can be classified into subjective image quality assessment methods and objective image quality evaluation methods.The latter has been widely studied in recent years because of the efficiency of evaluation and the lower cost of image quality assessment(IQA).Most of the objective image/video quality evaluation methods are based on manually extracted valid features,which come from the disparity map,depth information and so on.Such methods may result in missing important features easily,or make extracted features flawed.On the other hand,in recent years,deep learning has been rapidly developed and successfully applied in speech recognition,image classification,text analysis,etc..One of its important features is,compared to manually extract features,the ability to automatically learn the comprehensive and detailed features.Under the guidance of this idea,this paper proposes a deep neural network for the automatic extraction of 3D stereo image features.In this depth neural network,(CRBM)is used to extract the initial image features,the top-level factorized third-order Boltzmann sets(FTO-RBM)are applied to the left and right image feature mixture training,and the multi-layer full-Layer neural network to obtain the estimated value and construct the 3D stereoscopic image quality evaluation model,and then fine-tune the entire depth neural network by backward propagation algorithm.Afterwards,this paper analyzes and debugs some important parameters of the depth neural network,and tests the model using LIVE 3D Phase II and IEEE-SA open library and 3D stereo image database based on single stimulus and pairwise comparison subjective quality evaluation method,The performance of the model was verified.Based on the above three-dimensional image quality evaluation model,this paper analyzes the improvement of the 3D image quality evaluation model and puts forward a new optimization algorithm based on iteration.The algorithm is based on the average pooling and feature weight distribution pooling method.The existing feature is optimized by traversing,and the 3D image quality evaluation model is constructed based on the support vector regression algorithm after updating the feature map.Then,the important parameters in the optimization algorithm are analyzed and debugged,and tested again using the above two public libraries.The results show that the optimized model has better performance than the optimized one and achieves the best3 D Stereo image quality evaluation level.After optimization of the model,the influence of the subjective evaluation quality on the model was analyzed based on the single stimulus and pairwise comparison method.In this paper,the influence of MOS model with different Gaussian noise on the performance of the model is analyzed by adding the Gaussian noise with different variance to the average observation value(MOS)obtained by the subjective evaluation method of single stimulus and pairwise comparison.In addition,in order to be able to adapt to the increasingly complex data volume,this paper makes GPU parallelization accelerated processing for the deep neural network used for feature extraction.In this paper,we focus on the parallel optimization of two parts: offline training and online testing.In the offline training phase,taking into account the advantage of the Python-based Theano library in automatic derivation and high modularity,this library is used to parallelize the optimization of the deep neural network,and the two public libraries are experimented.Made a detailed comparison and analysis.In the online testing phase,considering the requirement of higher performance,we use CUDA to parallelize the steps of the testing process.The analysis of the kernel function and the thread allocation is given,which makes the acceleration performance further improved.