The Research Of Heterogeneous Image Patches Matching Based On Residual Feature Maps Learning And Image Conversion

Image matching has always been one of the emphases of research in the field of computer vision,and has been widely applied in many fields such as image retrieval,image registration and target tracking.Heterogeneous images contain a wealth of complementary information,which is very crucial for the comprehensive and accurate recognition of the target.However,due to different imaging mechanisms and conditions,there are obvious characterization differences between heterogeneous images,which makes the matching method of homologous images not applicable to heterogeneous images.Therefore,it is very important to study the matching methods of heterogeneous images.With the rapid development of deep learning,image matching methods based on convolutional neural networks emerge unceasingly.On the basis of summarizing the existing image matching methods,this paper applies the deep convolutional network and the generative adversarial network(GAN)to the matching of heterogeneous images.The main research work is as follows:First,heterogeneous image patches matching method based on feature map fusion.Aiming at the problem that the expressivity of the feature information extracted from heterogeneous image is insufficient,the spatial neighborhood information is introduced into the image feature descriptor through feature map fusion,and the valid information of the image is fully extracted.In order to solve the problem that the discrepancy information in the image descriptor is easily neglected,this paper improves on the feature map fusion method.In this way,the residual feature map is obtained by using the difference between feature maps,and then image patches are compared accordingly.The strategy not only enhances the network's attention to the key matching features,but also increases the sparseness of the network itself.As a result,the image patches matching accuracy and network robustness have been improved.Second,heterogeneous image patches matching method based on residual feature map learning.Through the analysis of the feature map fusion method based on their difference,it is found that the information on the difference features is very important in image matching.Aiming at the problem that some important detail information is missing in the high-level residual feature maps,the learning of multiple scale residual feature maps obtained from different convolutional layers in the backbone network is performed by adding a sub-branchconvolution network.The strategy integrates the underlying detail residual features and the high-level semantic residual features to fully exploit the key feature information in the image,which is instrumental in image patches matching.On this basis,this paper further combines the high-level residual feature maps obtained from the backbone network and the sub-branch network,and uses the three-measurement method to improve the accuracy and generalization performance of the matching.Third,heterogeneous image patches matching method based on image conversion.Aiming at the problem of characterization differences between heterogeneous images,the generator in GAN is used to convert the heterogeneous images into the homologous images,and then similarity measures are performed on homologous images to reduce the difficulty of matching.In order to improve the quality and diversity of the generated images,the pretrained convolution auto-encoder is used to regularly constrain the training of the discriminator network in GAN,thereby indirectly enhancing the stable training of the generator.The strategy of image conversion overcomes the difficulty of matching between images due to the considerable characterization differences and provides a new solution for heterogeneous image matching.