Research And Application Of Multimodal Fundus Image Registration Method

Fundus image registration is to complement and synthesize the information expressed by fundus images of different fields of view,different time and different modalities,which can improve the accuracy of ophthalmic disease diagnosis.However,due to the factors such as illumination,turbidity of the refractive medium,and operator experience,the acquired fundus images have problems such as uneven illumination,low contrast,and high noise.Furthermore,since the small overlap among images taken from different angles and inconsistent features of different modalities,it is challenge to find features or insufficient matching to achieve fundus image registration.Therefore,a high-success rate,high-precision,and high-efficiency fundus image registration method is proposed in this paper which is based on the most stable and prominent vascular structure in fundus images,combined with the advantages of deep learning in image feature extraction and outliers rejection.In order to obtain a complete and clear vessel image,the brightness and contrast of the fundus images is first improved and the noise is reduced.Then,the improved DNNs（Deep Neural Networks）vessel segmentation network is used for vessel segmentation of both CF and FA images.In this paper,the research work is carried out as follow:（1）A D2net-Reject registration method based on vessel images is proposed,which is improved in both feature extraction and feature matching.The D2-net feature extraction method based on deep learning is adopted to extract high-dimensional feature information while extracting low-dimensional feature information,which can solve the problem of insufficient useful feature points extracted by traditional SURF method.A matching method based on average distance is proposed and compared with the coarse matching algorithm based on Lowe’s distance,the proposed method can generate the best matching pair screening threshold according to the characteristics of the input image pairs,which can improve the accuracy of the coarse matching.Furthermore,a refine matching network is adopted and compared with the Random Sample Consensus algorithm,the adopted method can increase the number of correct matching pairs and improve the accuracy of screening matching pairs.（2）Based on the above work,a D2net-Flow registration method based on vessel images is proposed.This method utilizes the complementarity of the parametric registration method（D2net-Reject registration method）and the non-parametric registration method（optical flow estimation network）.And the non-parametric registration method is adopted to perform fine registration on the results of the parametric registration method registration,which can improve the registration accuracy on poor quality datasets.The proposed registration methods are applied to the private dataset of multimodal fundus images,the public dataset of multimodal fundus images,the public dataset of monomodal fundus images,and the stitching public dataset of fundus images,and compared with the traditional and deep learning registration methods.The experimental results demonstrate that the D2net-Reject and the D2 netFlow registration methods based on vessel images can ensure a 100% registration success rate on the public datasets of multimodal and monomodal fundus images.The registration accuracy is 2.11% and 11.16% higher than the highest accuracy in the comparison method,respectively.For the private dataset of multimodal fundus images,the registration success rates of the two methods both reach 93.46%,and the registration accuracy is improved 2.75% and 50.71% respectively.The registration accuracy of the D2net-Flow method has been significantly improved.In addition,the registration success rate on the public stitching dataset of fundus images is as high as96.4%.It is shown that the proposed registration method is not only suitable for monomodal and multimodal fundus image registration,but also for fundus image stitching,which can effectively improve the registration success rate and registration accuracy.In addition,it also has broad application prospects in other registration fields.