Image Stitching Based On Deep Learning

Today,computer vision tasks have always been a research hotspot in the field of artificial intelligence.Mining the deep information contained in images can help us better understand the world.Image stitching is the basis of many computational vision tasks,and it has a wide range of applications,such as scene rendering in virtual reality,camera ultra-wide viewing angle image synthesis,satellite debris image combination,and so on.However,there are still many challenges in the process of image stitching.The most important problems are the followings: the low quality of the extracted image features will affect the following steps of stitching;the lack of robustness of the feature points elimination algorithm,such as RANSAC algorithm,will lead to the residual of wrong matching point pairs;image distortion during the splicing process will affect the overall splicing performance.This paper proposes some solutions to above three problems.The main research contents and innovations are as follows:(1)Preliminary feature extraction network based on channel attention and anti-aliasing.Weak texture and low-light images often lack of sufficient features,and it is difficult for traditional algorithms to extract effective information from them.In order to solve this problem,this paper proposes a preliminary feature extraction network based on the improved Siamese network framework,and incorporates the channel attention module and anti-aliasing module to enhance the attention and anti-aliasing capabilities of the network,thereby improve the overall network performance and feature extraction capabilities.At the same time,in order to quantify the performance of image splicing better,this article also proposes a new quantification method.(2)Compound feature extraction network based on attention model.After the matching relationship is established between the feature points,it is necessary to use algorithms such as RANSAC to eliminate the wrong matching point pairs,but the performance of the RANSAC algorithm is limited by the selection "inter point" threshold.In order to strengthen the robustness of the algorithm for eliminating mismatched points,this paper proposes a selfattention module and a compound feature extraction network based on the attention mechanism.The self-attention module generates an "inter point" probability map through learning to highlight the content of the feature map that has a greater contribution to the estimation of the homography matrix;the compound feature extraction network is based on the non-local module and the light attention module to enhance the network's attention capacity and effective receptive field.(3)Deformation resistant module based on deformable convolution and STN.The homography matrix is used to warp the source image and merge it with the target image to form the final stitching result.However,the deformation of the source image will have a significant impact on the homography matrix estimation and the stitching result.In order to strengthen the anti-deformation ability of the network,this paper proposes an anti-deformation module based on deformable convolution and STN,which strengthens the network to learn the deformation in the source image,and at the same time allows the network to have sufficient translation,rotation and scaling invariance.Finally,this paper also designed a new loss function to accelerate the training of the network and strengthen the generalization ability of the model.