Research On Image Super-Resolution

Images are the main carrier of visual information,and their resolutions determine the richness and granularity of visual information.Due to the practical limits in real-world applications like manufacturing costs,power consumption,transmission bandwidth,and imaging conditions,original images obtained by a camera may not meet the requirements.Missing details in low-resolution images decrease the visual quality and limit the performance of downstream image processing tasks.To address this issue,image superresolution(SR)has been proposed to restore high-resolution images from low-resolution ones.As a classical low-level computer vision problem,image SR is of great research values and still a challenging task.This thesis presents a technical investigation on single image SR,stereo image SR,video image SR,hyper-spectral image SR,and image SR network acceleration.The main contributions are summarized as follows:(1)Single image SR: First,to achieve image SR under multiple degradations,a single image SR method using degradation representation learning is proposed.The proposed method leverages unsupervised contrastive learning to learn degradation representations to obtain degradation information rather than explicitly estimates the degradation.Second,to achieve image SR with multiple scale factors,a scale-arbitrary single image SR method is proposed.The proposed method uses conditional convolutions to generate dynamic kernels based on scale factors and achieves scale-arbitrary SR with a single model.(2)Stereo image SR: First,to extract correspondence between a stereo image pair,a parallax attention mechanism is proposed by combining epipolar constraints with selfattention mechanism.The parallax attention mechanism does not rely on a maximum disparity and can capture dense correspondence along epipolar lines with small computational and memory cost.Second,a parallax attention based stereo image SR method is proposed.The proposed method use stereo correspondence captured by the parallax attention mechanism to aggregate complementary information from left and right images to achieve stereo image SR.(3)Video SR: To improve the temporal consistency in video SR,a video SR method using high-resolution optical flow estimation is proposed.The proposed method integrates the SR of spatial information in a separated frames and temporal information between adjacent frames into an end-to-end framework,and leverages super-resolved temporal information to aggregate multiple frames to produce better temporal consistency.(4)Hyper-spectral image SR: To fuse spatial information and spectral information,a Transformer based hyper-spectral image SR method is proposed.The proposed method incorporates endmembers in a neural network for explicit modeling and uses endmembers as a connector to aggregate spatial and spectral information for hyper-spectral image SR.(5)Image SR network acceleration: First,to handle redundant computation in image SR task,an acceleration method using sparse mask convolutions is proposed.The proposed method learns binary spatial and channel masks to localize redundant computation in a SR network and then uses sparse mask convolution to dynamically skip these computation for acceleration.Second,to improve the efficiency of online activation quantization,a neural network quantization method using learnable lookup tables is proposed.The proposed method constructs learnable lookup tables as quantization function and leverages a simple lookup operation to achieve efficient online quantization of activations during inference.