| With the rapid development of remote sensing imaging technology,optical sensor and synthetic aperture radar(SAR)have become the important means of earth observation system.Due to the different imaging principles,the images generated by the two sensors are highly complementary,which play an important role in both military and civilian fields.In order to make full use of the complementary information of optical image and SAR image,multi-source images of the same scene need to be registered and fused.However,due to the different imaging mechanisms of the two sensors,there are large differences in phase and resolution between the images obtained.Therefore,how to achieve high precision registration and high quality fusion of multisource remote sensing images has become the focus of multi-source image collaborative application.In this paper,based on Sentinel-1 satellite data,Sentinel-2satellite data and Sen1-2 data set,the registration and fusion of optical and SAR images are deeply explored.The specific research work and innovation results are as follows:(1)Image registration method based on traditional and deep learning feature.For Sen1-2 optical and SAR image data sets,Optical SAR SIFT(OS-SIFT)algorithm,Histogram of Orientated Phase Congruency(HOPC)algorithm,Deep Learning(DL)technology with Improved Harries(DIH)algorithm,Description and Detection Net(D2-Net)algorithm are respectively used for image registration experiments.By using Root Mean Squared Error(RMSE),Correct Match Ratio(CMR)and logarithm of matching points as evaluation indexes,the four algorithms are evaluated.Among them,the registration result of HOPC algorithm is relatively good.The average RMSE is 1.368,but it still could not meet the requirements of high precision registration.(2)Image registration method combining similar edges and OFAST-BRISK feature extraction.Aiming at the problems of low accuracy and poor stability in current multi-source remote sensing image registration,this paper proposes a new optical and SAR image registration method--DL Edge Feature with the OFAST-BRISK(DEF-OB)which combines the advantages of deep learning and traditional feature algorithms.Firstly,an improved Log operator and an improved Deep Labv3+ network is used to extract similar edge features from optical and SAR images.Then,use the OFASTBRISK algorithm to detect and describe the feature points.Finally,the improved Random Sample Consensus(RANSAC)algorithm is combined to achieve accurate registration of optical and SAR images.Compared with the existing algorithms,the registration results of multi-source remote sensing images proposed in this paper are significantly improved in terms of RMSE,CMR and other parameters,and the average RMSE is as low as 0.697.In addition,the DEF-OB method also achieves good results in the registration of rotating images,which verifies the superiority of DEF-OB method.(3)Fusion method of optical and SAR remote sensing images based on texture features.Aiming at the problems of inadequate feature extraction and serious noise influence in multi-source remote sensing image fusion,an image fusion method based on deep learning texture feature extraction is proposed.Firstly,the texture information of SAR images is extracted by gray co-occurrence matrix combined with VGG-19 deep learning network.Then,HIS transform and wavelet change are used to extract the high frequency detail components of optical images,and SAR image texture features are used to enhance the high frequency detail components.Finally,the fusion image is obtained by inverse wavelet change and inverse HIS transform.For the images before and after fusion,support vector machines are used to classify ground objects respectively,and the overall classification accuracy is improved from 90.42% to93.97%.Kappa coefficient increase from 0.863 to 0.912.Based on the research of traditional feature algorithms and deep learning-based optical and SAR image registration methods,this paper proposes an optical and SAR image registration method that combines similar edges and OFAST-BRISK features.It solves the problems of low registration accuracy and low robustness existing algorithms.In addition,texture feature based fusion method is used to fuse the images registered by optical and SAR and classify the surface objects.The experimental results show that the registration and fusion method of optical and SAR remote sensing images proposed in this paper can effectively improve the accuracy of surface object classification of optical images,and provide important support for the collaborative application of multisource remote sensing images. |
