Research On Image And Video Super-resolution Towards Complex Scenarios

As a key technology in the field of image processing,image and video superresolution(SR)has important theoretical significance and application value in many fields because it can improve image and video resolution while enriching their details,which has been widely concerned by academia and industry.In recent years,with the rapid development of deep learning theory and convolutional neural networks,deep learningbased image and video SR reconstruction have become a research hotspot.However,in practical applications,image and video SR technology is often faced with various challenges in complex scenarios,such as limited computing resources,diverse and even unknown degradations.In the face of these challenges,how to quickly,efficiently and accurately reconstruct high-resolution(HR)images and videos from low-quality and lowresolution(low-resolution)images and videos is an urgent problem to be solved.This dissertation researches deep learning-based image and video SR methods towards complex scenarios and practical applications.By carrying out research on real-time reconstruction,high-quality detail recovery,model generalization ability,and HR video generation in SR technology,we put forward a series of image and video SR methods.The main research works of this dissertation are summarized as follows:·For the real-time requirements in practical applications,this dissertation proposes an adaptive information filtering network(Filter Net)for image SR.The proposed Filter Net uses dilated convolution and residual learning as the basic components to improve the receptive fields of shallow deep networks for effective image context information capturing,so as to achieve a fast reconstruction and ensure promis-ing performance.In order to further improve the detail recovery ability of the network,a gate selection mechanism(GSM)and an adaptive information fusion structure(AIFS)are designed,so that the network can adaptively focus on mean-ingful high-frequency information and filter redundant low-frequency information.Experiments on public datasets demonstrate the superior performance of the pro-posed method in terms of both reconstruction speed and texture accuracy.·Aiming at the problem of high-quality detail restoration in image SR,this disserta-tion proposes a deep interleaved network(DIN)which adopts a multi-path and mul-ti branch framework to effectively fuses and enhances the information of different states,thus improving the feature representation ability of networks.Firstly,this dissertation organically combines local residual connection,dense connection and deep separable convolution to construct a weighted residual dense block(WRDB) to realize more accurate feature aggregation and propagation in the way of multi-path fusion.Then,on the interleaved nodes of different branches,an asymmetric co-attention(Asy CA)mechanism is proposed to adaptively emphasize the features from different branches,which can facilitate the recovery of high-frequency de-tails.Experimental results on different degraded images show that the proposed method can significantly enhance image details while enlarging image resolution,demonstrating the robustness and generalization of this method.·For the complicated and unknown degradations in real-world scenarios,this disser-tation proposes an effective detail-structure alternative optimization network(DSS-R)to tackle the blind SR problem from image detail and structure perspectives.The core of DSSR is the detail-structure modulation module(DSMM)composed of a detail restoration unit and structure modulation unit.By exploiting the interaction and collaboration of image details and structural contexts,we can achieve the alter-native optimization of image details and structures.At the same time,this disser-tation constructs the SR framework from the recurrent convolution neural network view.In this way,the network can achieve iterative and alternative optimization across time.Experiments indicate the effectiveness of the proposed method in mul-tiple scenes,such as synthetic datasets,unknown degraded images in the real world and surveillance images et al.,demonstrating the excellent generalization and even universality.·For the video super-resolution(VSR)problem,this dissertation presents a deep dual attention network(DDAN),which includes a motion compensation network(MCNet)and an SR reconstruction network(Recon Net),to make full use of spatio-temporal information for accurate VSR reconstruction.The MCNet progressively learns the optical flow representations to synthesize the motion information across adjacent frames in a pyramid fashion for accurate motion estimation and motion compensation.Besides,in order to alleviate the mis-registration errors caused by motion estimation,we extract the detail components of original LR adjacent frames as supplementary information to assist the subsequent HR video reconstruction.In Recon Net,this dissertation combines the channel-wise attention mechanism,spatial attention mechanism and residual learning to construct a residual attention block(RAB),which enables the network to focus on meaningful features along channel and spatial dimensions for high-frequency details recovery.Experimental results on a large number of datasets demonstrate the superiority of this method in terms of quantitative and qualitative assessment.