Research On Deep Learning-based Inter Prediction Enhancement For H.266/VVC

The enormous progress of the information age has led to an increasing demand for high-definition video.The ever-increasing amount of video data requires higher and higher video encoding efficiency.The latest generation of video coding standard,H.266/VVC,has been released.Compared with the previous encoding standard H.265/HEVC,H.266/VVC achieves about a 50%improvement in encoding efficiency under the same subjective quality conditions.In recent years,deep learning has been widely applied in image processing,computer vision,and other fields due to its excellent data fitting capabilities.This thesis aims to combine deep learning methods with the inter-frame prediction module in the H.266/VVC encoding framework to further improve video encoding efficiency.The main work and innovation points of this thesis are as follows:(1)This thesis proposes a method of using neural networks to enhance the quality of the luminance component of reference blocks.The method first extracts different-sized prediction blocks and original blocks from the bitstream file and the original video signal to create a dataset,and trains multiple network models to enhance the quality of different-sized reference blocks before motion compensation,in order to restore the image information loss caused by quantization as much as possible.By improving the quality of reference blocks,the quality of the current block reconstruction is enhanced,thereby improving encoding performance.(2)This thesis proposes neural networks based on attention mechanism and dense connections to enhance the quality of reference blocks.When the image texture is complex,it is difficult for the network model to enhance the image based on image content.The introduction of attention mechanism can better enhance the quality of the image according to the content of the reference block,thereby improving the encoding performance.To address the issue of large network model parameters,a dense connection neural network is proposed,which achieves similar encoding performance as the previous neural network model with fewer parameters.(3)This thesis proposes a reference frame enhancement method using neural networks to improve the efficiency of inter-frame prediction coding.The method enhances the overall quality of the reconstructed reference frame.Compared with the reference block enhancement method,this method has lower encoding and decoding complexity.Moreover,in view of the negative gain of the network model on some images,a flag is added to the Coding Unit to determine whether the network model is used for enhancement.Finally,the performance of the proposed method is verified through experiments.