Multi-Path Convolutional Neural Network With Cooperative Adversarial Learning For Image Super-Resolution

Image super-resolution(SR)reconstructs SR image by directly extracting features from low-resolution image,which is widely used in computer vision area.In recent years,deep convolution neural SR networks have recently improved SR image quality because many deep network structures are designed to improve feature representation,and different loss functions assess image quality from different aspects to provide constraints for deep learning.For example,cascaded multi-path network is applied to multi-scale image SR reconstruction,adversarial loss and feature loss help to reconstruct visually pleasing SR image in generative adversarial network.However,there are some limitations among these multi-path networks: The cascaded structure of multi-scale network can not fuse multi-level deep features.Hard-trained and time-consuming feature loss network or discriminator in generative adversarial network is useless in reconstruction process.The reconstructed visually pleasing SR image often gets very low scores of image quality assessment(IQA).To improve feature representation,we firstly propose a multi-level residual features reused network(MRRN).Analysis of multi-scale receipt field and visualization of different level feature maps reflect that the MRRN helps to enhance feature representation.To mitigate these issues of multi-path network,we further design a cooperative adversarial network(CAN)for SR,which consists of two SR sub-networks(named as chaser and enhancer respectively).On cooperative side,enhancer is cascaded after chaser and minimizes average pixel error between the SR image of chaser reconstructed and the original high-resolution image.On adversarial side,chaser maximizes the feature loss provided by enhancer,but enhancer minimizes it during training.In order to merge Chaser and Enhancer into a unified SR network,we propose an alternatively iterative training method of three training units.We finally train a CAN model using residual network as chaser and the MRRN as enhancer.Compared to existing multi-network models,our CAN reconstructs visually pleasing SR image of high IQA scores with less parameters.