Research On Stereo Image Super-resolution Based On Deep Learning

In stereo vision,stereo SR is a low-level vision task that uses compensation information between left and right views to increase image resolution and improve image visual effects.Deep learning has great advantages in extracting image features and contextual information,is widely used in image processing.In this paper,we focus on the application of deep learning in stereo SR,trying to improve the representation ability of convolutional neural networks and reducing model complexity.The main work of this paper is as follows:Firstly,we expound the basic modules of CNN in deep learning and disparity,epipolar constraints,stereo matching in stereo version to clarify the task requirements of Stereo SR based on deep learning.The mainstream algorithms in this field are studied,especially the basic ideas and applicable scenarios of algorithms based on parallax attention mechanism and stereo matching.Secondly,the classical stereo SR algorithms are tested under the same training set,which confirms the performance superiority of the stereo matching algorithm,and discuss the optimization direction of the algorithm based on the parallax attention mechanism.Then,by comparing training results of PASSRnet under different datasets,we confirm that the diversity of Flickr1024 and Holopix50 K datasets can achieve higher reconstruction accuracy while alleviating the problem of overfitting.Subsequently,we propose MSR-SAMnet to extract complementary information between views and ensure the integrity of boundary information.We first introduce a multi-scale residual feature extraction block to realize multi-scale extraction.Then,The disparity information along the epipolar line is captured by SAM.Meanwhile,cascaded residual dense blocks are used in the reconstruction module to improve the effective transfer of reconstructed features while expanding the receptive field.Compared with PASSRnet,0.15 d B PSNR and 0.1d B SSIM improvement are achieved,which confirms the effectiveness of the network in extracting inter-view interaction information.Finally,we propose MMF-LSRnet to solve the trade-off between reconstruction accuracy and computational overhead in stereo SR network.Referring to the lightweight single-image superresolution network,we use a modified binary feature fusion architecture to fully extract in-view information.Based on the design of LPAM,DCPAM and the pyramid sampling mechanism are used to improve the accuracy of stereo matching.Among them,the network based on LPAM can achieve a similar reconstruction effect to PASSRnet,while DCPAM increases 0.2d B PSNR with only an increase of 20 K parameters.The network achieves excellent reconstruction results with the least computational cost in the current stereo SR module,which proves the portability of the lightweight network in the stereo SR task.