An Enhanced Video Compression Framework Using A Content-fitted Restoration Neural Network

Nowadays,videos accounts for most of the world's internet traffic about 59%,and this proportion will increase to nearly 79%by 2022,which brings a great challenge for video transmission and storage.Therefore,video compression is indispensable and its research has never stopped.In the past few decades,although classic video compression standards such as MPEG,H.264/AVC,H.265/HEVC,etc.have become more mature,there is still a desire to further improve the compression performance in the face of the huge cost of transmission and storage.At the same time,the deep learning algorithms represented by convolutional neural networks have developed rapidly in recent years.They have obtained remarkable results in combination with many fields,and even have the tendency to replace traditional algorithms.Therefore,the application of deep learning tools to the field of video compression so as to improve the efficiency of video compression has attracted widespread attention.More and more related work is in progress,including the restoration of decoded video.Many existing deep learning based video compression works apply the DNN to restore decoded videos by learning the mapping between the decoded video and raw video(ground truth).The big challenge is to train a well-fitted model(one mapping)for various video sequences.Unlike the applications such as image enhancement whose ground truth can only be obtained in the training process,the video encoder can always get the ground truth which is the raw video.It means that we can train the model together with video compression and use one model for each sequence or even for each frame.Based on the above background,we propose an enhanced video compression framework using a content-fitted restoration neural network(EVC-CR)specifically for video on demand(VOD)applications in this paper.At the encoder side,a lightweight content-fitted restoration neural network(CRNN)is trained for a group of consecutive frames,so that it is well-fitted to this group and achieves a strong restoration ability.After that,the learned parameters of CRNN are transmitted to the decoder with the encoded bitstream.At the decoder side,CRNN can perform the same robust restoration operation on the reconstructed frames.First,the proposed EVC-CR outperforms other DNN-based restoration algorithms in terms of compression performance.It achieves the highest compression efficiency with less computational complexity.Second,the proposed EVC-CR is compatible with of all existing video codec systems without modifying their kernels.Experimental results show that the compression performance can be greatly improved in terms of 5.891%-24.413% BD-BR reduction,by integrating CRNN with VTM,HM,X265,X264,and the DNN codecs DVC.This allows the H.264 and H.265 codec IPs developed with a lot of effort to be reused.Finally,the really-time decoding for 1080 p 30fps can be satisfied both on the server equipped with Nvidia 2080 Ti GPU and on the AI edge device Nvidia Jetson TX2.