Compressed Video Post-processing Based On Convolutional Neural Network

With the progress of science and technology,the video tends to high-definition,ultrahigh-definition,high frame rate.A lot of video-based business applications continue to emerge in recent years at the same time.The development of video technology and the emergence of new business applications bring convenience to our lives,but the surge of video data brings great challenges to the data storage or network transmission.Video or image coding standard is a technique to alleviate the contradiction between the scarcity of infrastructure and the high quality video demand.But the quality of compressed video will decline.Therefore,improving video quality without increasing the amount of data has become a real demand.The way to realize this demand is the compressed video post-processing.Moreover,it is not contradictory between the efficiency improvement of the coding standard and the post-processing of compressed videos.Both of them are important directions of video technology research.In this paper,convolutional neural network is used in the compressed video post-processing,and the main research achievements are as follows:(1)Frame enhancement algorithm based on convolutional neural network.This paper proposes a self-learning based video enhancement scheme using convolutional neural network in order to cope with the frame distortion caused by compression.Learning an end-to-end mapping between the compressed and original frames using convolutional neural network in the encoder.Finally,the high quality video sequences can be reconstructed with low quality video sequences and neural networks in the decoder;(2)Frame rate up conversion algorithm based on convolutional neural network.Frame rate up conversion algorithm is mainly used to increase the video frame rate to eliminate the blur and jitter problem in low frame rate video sequence,so as to enhance the visual quality.In order to achieve this purpose,this paper presents a self-learning based frame rate up conversion scheme based on convolutional neural network.The scheme uses two adjacent frames to predict the interpolated frames by convolutional neural network.Compared to the traditional block matching based method,this scheme can effectively avoid the holes effect and occlusion effect benefited from it is based on global prediction;(3)Asymmetric stereo video compression post-processing algorithm based on convolutional neural network.Asymmetric stereo video compression is a widely used video compression method in multi-view video coding.The quality of low quality view video is relatively poor although the compression efficiency has been improved.In view of this situation,this paper proposes a scheme for post-processing of low quality view video using the high frequency information of high quality view video.In this scheme,the macroblock in the low quality view video frame is searched for the matching macroblock in the corresponding high quality view video frame,and then it makes the most similar macroblocks into a macroblock group.Finally,this scheme reconstructs the high frequency information of the low quality view video with the macroblock group.