Temporal&Spatial Information Fusion Neural Network Based Research For In-Loop Filter

Recently,high definition video technology brings about more satisfactory visual experience to consumers.Meanwhile,the large volume of video data also causes high transmission and storage burden.Therefore,video coding technology is increasingly important for relieving the burden.The conventional video coding technique is based on transformation and quantitation operations,so it is lossy compression.So,in the proceeding of compression,some detail and high frequency information would be lost.As a result,there comes out the blocking,ringing and blurring effects in compressed videos,which severely influence visual satisfaction.In conventional video compression technology,the quality restoration technique exists which is called in-loop filter.But the generalization ability of in-loop filter technique is limited,so it can't achieve good restoration results for different videos all the time.Based on these challenges,we employ deep learning models to learn different quality distortion patterns from big video data and generalize it to different test datasets.Meanwhile,the deep learning model has both high compression ratio and better filter results.First of all,for quality distortion occurs in single frame,we employ deep neural network to enhance model generalization ability.Since recent proposed residual learning and dense network can drastically improve model depth,we combine two techniques together to construct our model.In another aspect,different videos may be shot from different sight of view.As for different scales videos,the model adopts different sizes kernels.In addition,the attention mechanism is also adopted to improve generalization ability.Furthermore,copious temporal and spatial information is contained in videos.Any frame in a video has information redundancy between it and its neighboring frames.Therefore,the neighboring frames to assist enhancing frame quality.Based on former mentioned model,we add temporal and spatial information fused network to utilize neighboring frames and similar patches.The multiple frame processing methods can further improve model performance.