Retinal Image Generation And Lesion Localization Based On Generative Adversarial Network

Retinal-related diseases are the main causes of vision loss.There are many types of medical images used for the diagnosis of retinal-related diseases,which can greatly help the clinical diagnosis of retinal-related diseases.With the rapid development of deep learning,significant progress has been made in image generation and biomarker localization.In the meantime,generative adversarial networks have been increasingly applied to the field of medical images.Considering the problems in the existing generation and localization models and the characteristics of retinal images,this paper has carried out research and application of retinal image generation and lesion localization based on generative adversarial networks.The specific work is as follows:(1)A texture-guided down-sampling and up-sampling model based on U-Net is proposed to generate optical coherence tomography angiography(OCTA)images from optical coherence tomography(OCT)images.By combining the extracted texture features and content adaptive convolution,a new texture guided sampling block is proposed.Then,the texture-guided U-Net is constructed by replacing the traditional convolution with textureguided sampling blocks.In addition,the Euclidean distance of image is used to construct the loss function,which can help the model to learn more useful similarity features between OCT and OCTA images.Compared with the most popular semantic segmentation models and generative adversarial networks,the results of cross validation experiments prove the stability and superior performance of the proposed model.(2)A joint optimization model is proposed to detect retinal diseases and locate the diseased area under limited training data.The joint optimization model combines the generative adversarial network and the classification network,and jointly optimizes them to achieve the optimal solution of multiple networks.A novel res-guided sampling block is proposed by combining learnable residual features and pixel-adaptive convolutions.A Resguided U-Net is constructed as the generator by substituting the traditional convolution with the Res-guided sampling blocks.Our model achieves superior classification and localization performance on three public color fundus datasets.Finally,a multi-modal joint optimization solution is proposed and applied to ultra-wide-angle fluorescein angiography images.The application results show that the model can accurately locate the leakage area and the nonperfusion area,showing strong clinical application value.(3)Based on the proposed feature-guided generator and joint optimization model,a retinal image generation and lesion locating system was developed.The core of the system is the calling and interactive interface design of the retinal image generation and disease locating model.The five functional modules of model selection,input image displaying,model training,model testing,and test image displaying make the process of retinal image generation and lesion locating more intuitive and easier to operate.A variety of configuration options are provided on the system interface,and models can be trained to adapt to real clinical application scenarios.