Compression And Post-processing Of Panoramic Video

With the increasing interest in virtual reality(VR),various commercialized virtual reality products are playing an increasingly important role in the market.Panoramic video has attracted much attention as the main content source of VR.Compared with ordinary video,panoramic video has the characteristics of high resolution,multi-perspective and large amount of data storage.In order to adapt to video transmission,we have to sacrifice the quality of panoramic video to save bit stream when we compress that using classical coding standards.Low-quality panoramic video seriously affects the user's immersive experience.Therefore,this paper proposes to apply Convolutional Neural Network to panoramic video compression and post-processing for effectively improving the quality of panoramic video.This paper mainly completes the following three tasks.(1)The quality of panoramic video is very important to the viewing experience of users,especially the video mapped from the panoramic video of a certain area to the current viewport.Subsequently,based on the Generative Adversarial Network of Convolutional Neural Network,Panorama-GAN network is proposed,in which the generation model is based on U-NET network,and the loss function is formed by calculating the difference of features by VGG19.In order to improve the quality of panoramic video,we use this network to train and generate models for compressed panoramic video and single view port with different mapping relations.Finally,experiments show that the network has a better performance in improving the video quality of the current viewport.(2)Considering the similarity between frames of panoramic video,this paper further proposes to encode panoramic video in cube format into two versions of high and low quality video.In the transmission process,a group of Pictures(GOP)is used as a unit to cross-transmit video.In the last step,a four-layer convolution neural network is used to extract and combine the features of low-quality frames and adjacent high-quality frames.Then the low-quality video frames are improved by feature enhancement,mapping and image reconstruction.Experiments show that our scheme improves the quality of the whole panoramic video sequence.(3)Panoramic video usually is transmitted based on scheme of viewport that high quality video stream of current viewport and low quality video stream of other viewports.Therefore,when users turn their heads,the network delay will cause the phenomenon of unclear turning pictures.Based on the solution of Tile compression panoramic video compatible with HEVC,this paper proposes a secondary compression scheme:on the server side,low-quality video is compressed by high-quality video again,or compressed after downsampling;on the client side,after receiving panoramic video,high-quality video at the current viewport is compressed and sent to convolutional neural network training.In the last,the generated model is used to Improve other low-quality video ports.The experimental results show the effectiveness of the scheme,and the downsampling scheme shows better performance in low bitrate transmission.