| With the rapid development of technologies such as the Internet and 5G,images and videos contents have become an indispensable part of life,and people’s demands for their quality are gradually increasing.The existing image and video compression standards are already mature enough to compress the original visual data at high rates for storage and transmission in the Internet.However,there are steps such as blocking and quantization in the encoding of compressed data,which leads to various distortions in the decoded visual data,such as block effects,artifacts,etc.,especially at high compression ratios,which seriously affects the viewing experience of images and videos.This thesis discusses the deep learning-based methods for enhancing the quality of compressed visual data,and designs a method for enhancing the quality of compressed images as well as two methods for enhancing the quality of compressed videos.(1)Dual-Branch Residual Network For Compressed Image Quality Enhancement.In this thesis,aiming at the problems of poor feature extraction ability and complex model of the existing compressed image quality enhancement networks,the residual network is used to design this algorithm,and the dual-branch residual block is used to improve it.Among them,the dual-branch residual block contains two parallel branches: a feature selection branch and a multi-scale feature extraction branch.The feature selection branch utilizes the channel attention mechanism to enhance features,while the multi-scale feature extraction branch increases the receptive field of the network by fusing multi-level feature information.The dual-branch residual block improves the compressed image quality by fusing the feature information of the two branches.(2)Compression Quality Perception Network For Compressed Video Quality Enhancement.Aiming at the problem that the existing single-frame method does not consider the quality fluctuation of compressed video,this thesis designs an algorithm from the perspective of quality estimation.This algorithm proposes a global quality estimation sub-network,which can estimate the quality of each frame,and use the quality estimation results as additional information to guide the compression quality enhancement network to enhance compressed frames.The compression quality enhancement network uses multiscale structure and depth separable convolution,which can reduce the inference cost of the model while fusing features.And,in the compression quality enhancement network,this thesis proposes a local quality attention module for salient features selection.Finally,this algorithm combines edge loss to improve high-frequency details in compressed frames.(3)Multi-Scale Deformable Alignment and Spatio-Temporal Attention For Compressed Video Quality Enhancement.Aiming at the problem of alignment errors in existing multi-frame methods caused by quality fluctuations in compressed frames,this thesis proposes an alignment method based on multi-scale deformable convolution,which aligns and fuses compressed video frames at the original scale and low scale,and introduce weights for spatio-temporal deformable convolutions to reduce errors.This thesis also proposes a new multi-scale spatio-temporal attention mechanism,further refines the alignment accuracy of the extracted features in the quality enhancement stage,and weights features in both spatial and channel dimensions to improve the stability of the model during training and the final performance.This thesis demonstrates the effectiveness of the proposed methods through quantitative analysis and visualization experiments on multiple datasets.Experimental results show that the proposed methods can significantly improve the objective and subjective quality of compressed visual data. |
