Research On Intra Prediction Coding Techniques Based On Deep Learning

The newest video coding standard is High Efficiency Video Coding(HEVC).Com-pared with H.264,HEVC achieves almost a half bitrate reduction at the same subjective quality.However,the modules involved in HEVC are derived via manual designs and optimizations.Hand-crafted designs usually rely on the premise of stationary signal.Since the characteristics of natural video do not meet the assumption,the optimality of those modules cannot be guaranteed.Deep learning can automatically learn feature extractors from training data and is suitable for solving supervised problems.Inspired by the remarkable success achieved by deep learning in different computer vision tasks,this dissertation makes use of deep learning to address partial issues from video intra prediction.In the first contribution,this dissertation proposes a deep learning-based block up-sampling technique for intra frame coding.The re-sampling-based technique,i.e.down-sampling before encoding and up-sampling after decoding,is a well known strat-egy to compress videos of high-resolution.However,previous studies on re-sampling based coding adopt hand-crafted filters for up-sampling,leading to a restricted perfor-mance.The first contribution includes the following novelties so as to improve the coding performance:This dissertation develops convolutional neural network-based filters for the up-sampling of luma and chroma components.This dissertation devel-ops an efficient solution to integrate the up-sampling CNNs into the intra frame coding framework.This dissertation proposes a two-stage up-sampling process,the first stage being within the block-by-block coding loop,and the second stage being performed on the entire frame,so as to refine block boundaries.This dissertation empirically studies how to set the coding parameters of down-sampled blocks for pursuing the frame-level rate-distortion optimization.In the second contribution,this dissertation proposes a deep learning-based block down-sampling technique for intra frame coding.Most of the existing down/up-sampling-based compression schemes adopt hand-crafted filters to decrease resolution,incurring the loss of important information.The second contribution includes the fol-lowing novelties so as to improve the coding performance:This dissertation defines image compact-resolution(CR)as the dual problem of image super-resolution(SR)in the context of compression.To guarantee that CR qualifies as the down-sampling filter,this dissertation proposes to impose two requirements on image CR.This dissertation proposes a convolutional neural network-based approach for image CR and translates the above two requirements of image CR into operable optimization targets for train-ing the CNN.This dissertation explores different training strategies as well as different network structures for the CNN.In the third contribution,this dissertation proposes a deep learning-based cross-channel prediction(CCP)technique for intra frame coding.The typically adopted tech-nique for CCP is Linear Model method(LM).Although LM is simple yet effective,it is oversimplified for complex image content or large blocks.The third contribution includes the following novelties so as to improve the coding performance:This dis-sertation proposes a hybrid neural network including both fully connected layers and convolutional layers for cross-channel prediction(CCPNN).This dissertation proposes to train CCPNN with a transform domain loss function,leading to a more compact representation of residue in the transform domain.This dissertation systematically in-vestigates the influence of hyper-parameters of the proposed network on the prediction performance.