Research For Stereo Image Super-Resolution Based On Deep Learning

Image super-resolution aims to reconstruct more high-resolution details from de-graded low-resolution images.With the development of binocular imaging technology,stereo images have been widely used in the fields of depth estimation,virtual reality and autonomous driving.Compared with single-image super-resolution,stereo images are highly symmetrical under the epipolar constraints caused by parallax,so the com-plementary information between the left and right views can be used to further improve the super-resolution performance.In recent years,researchers have successfully applied deep learning to stereo image super-resolution.Most of these methods use disparity prior to obtain matching information between images and achieve excellent performance,but there are still some problems: 1)The feature mining and fusion of the two views are still insufficient,and 2)the ability to model variable disparity between stereo images is still not effective enough.To handle these problems,this paper has carried out the following work:(1)A cross-scale and cross-view non-local feature based stereo image super-resolution network(CSCISSRnet)is proposed.In order to solve the problem that the two-view fea-ture fusion is not effective enough,a residual feature fusion module(RFFB)is proposed,which makes the network focus on the different information between the feature sources and improves the representation ability of the fused features.In addition,to solve the problem of insufficient feature mining between two views,the local and global context in-formation within and across views is fully utilized,and a cross-scale cross-view non-local attention(CSCIN)mechanism is proposed to directly obtain high-resolution features with cross-image similarity.The various features excavated in the network are continuously fused through RFFB to enhance the utilization of information within and across views.(2)Based on the closely intertwined and mutually boosting characteristics of stereo image super-resolution and disparity estimation,a stereo image super-resolution and dis-parity estimation feedback network(SSRDE-FNet)is proposed,which uses high-precision disparity to guide the left and right images to obtain more accurate correlation and facilitate super-resolution,enabling joint optimization of the two tasks in an end-to-end neural net-work.SSRDE-FNet estimates high-resolution disparity in a coarse-to-fine manner,firstly,it uses low-resolution disparity to guide the stereo super-resolution reconstruction,then the generated high-resolution stereo features are used to construct and aggregate matching costs to achieve high-resolution disparity estimation.Subsequently,a high-resolution dis-parity information feedback(HRDIF)mechanism is proposed,which includes two strate-gies: aggregated high-resolution feature feedback(AHFF)and low-level feature represen-tation enhancement(LRE).The mechanism feed the disparity information back to previ-ous layer and further improve the performance of the two tasks by network recursion.To verify the effectiveness of the proposed methods,extensive experiments are con-ducted on public datasets.The results show that compared with other models,the proposed methods have advantages in both quantitative indicators and visual effects.Furthermore,ablation experiments verify the effectiveness of each module and strategy proposed in this paper.