End-to-end Video Coding Method Based On Convolutional Neural Network

Video coding technology plays an important role in people's daily life,especially in video transmission and storage.In recent years,deep learning has achieved great success in computer vision and other fields.Some deep learning-based methods have also been proposed in the field of video coding.These methods are designed to achieve more efficient video compression and encoding based on a deep neural network-based video coding framework.At present,the related research is still in its infancy,and there are many issues that need further research.One of the important issues is to study an efficient convolutional neural network implementation method for motion estimation and motion compensation in traditional hybrid coding schemes.On this issue,there is a lot of room for improvement in the current method.For example,consider bidirectional multiframe prediction and compensation,and consider YUV format based on human visual characteristics.Therefore,this paper studies these problems and proposes an end-to-end video compression coding method based on deep learning.The specific research results are as follows:1.An end-to-end video compression coding framework considering bidirectional multi-frame motion vector prediction is proposed.In this framework,a motion estimation neural network based on convolutional neural network is constructed.For the motion estimation module,instead of encoding the real motion vector directly,the prediction difference is encoded using an auto-encoder to reduce the encoding complexity.On the other hand,a bidirectional multi-frame compensation network is also constructed.In this network,the edge images of four bidirectional video frames are jointly used,and the basic idea is to determine the position contour of the object through the edge.The quality of reconstructed video is improved by restricting the edges of the compensated frame to reduce artifacts in the compression algorithm.2.An end-to-end video compression neural network considering the visual characteristics of human eyes is proposed.Due to the different sensitivity of human eyes to luminance and chrominance,the proposed method is based on the YUV color space to construct more targeted compression coding sub-networks for different channels.For the chrominance channel,a relatively coarse motion compensation network is constructed;for the luminance channel,a finer multi-level motion compensation network is constructed,and chrominance information is incorporated to capture more accurate luminance information.3.A suitable training set is constructed based on the commonly used video coding test sequences,and the two proposed methods are tested and analyzed on the real data set JCT-VC.The results show that,compared with several related methods,the proposed method can obtain reconstructed videos with higher PSNR at lower or roughly the same BPP.Moreover,the more general R-D curve analysis also shows the advantages of the proposed method.