Research On Stereoscopic Video Quality Assessment Based On Convolutional Neural Network

In recent years,the development of 3D processing technology has prompted the commercialization of stereoscopic video.However,any 3D processing technology will cause the quality of stereoscopic video decline,stereoscopic sense deficiency and other issues,thus affecting human viewing experience.Therefore,how to build a stereoscopic video quality evaluation model according to the characteristics of stereoscopic video which better conforms to human vision has become one of the key issues in the field of computer vision.At present,the quality evaluation methods of stereoscopic video mainly include feature extraction based method,sparse representation based method and deep learning based method.Convolutional neural network,as an important technology theory of deep learning,has made extensive achievements in the field of computer vision.Therefore,this paper mainly carries out two aspects of work based on convolutional neural network.Firstly,a fast no reference stereoscopic video quality assessment method via analysis of binocular-fused key frames is proposed in this paper.Binocular fusion algorithm is firstly used to fuse stereoscopic video into fusion video,then key frames are extracted by key frame extraction algorithm from fusion video.Finally,key frame features are mapped to stereoscopic video quality score by fine-tuning convolutional neural network.This method only analyzes the features of key frames,which greatly shortens the test time of the algorithm and solves the time-consuming problem in the current stereo video quality evaluation methods.Secondly,a stereoscopic video quality assessment based on the two-step-training binocular fusion network is proposed in this paper.This method proposes a new binocular fusion network,which simulates the complex fusion and processing process found in the entire visual path from binocular stereoscopic information acquisition to stereo image quality judgment.Secondly,a two-step-training strategy is adopted for the proposed binocular fusion network.The local network is firstly trained with patches scores and then the global network is trained with MOS or DMOS values.This two-step-training strategy solves the problem of poor evaluation results of asymmetric distortion in the current stereoscopic video quality assessment methods.In addition,the input of the proposed convolutional neural network are spatiotemporal saliency feature flows rather than the original stereoscopic videos.In order to verify the reliability and validity of the above stereoscopic video quality assessment methods,experiments were carried out on two public stereoscopic video databases,NAMA3DS1-COSPAD1 and QI-SVQA.The experimental results show that the two objective quality assessment methods have a good correlation with subjective scores,and have good results in symmetric distortion and asymmetric distortion stereoscopic video,which have a strong universal performance.