Key Technologies Research For Ophthalmic Multi-Disease Diagnosis Based On Transformer

Currently,there are over 418 million people worldwide suffering from sightthreatening ophthalmic diseases.Deep learning-based automatic grading technology for ophthalmic diseases can help patients timely detect eye health issues and receive prompt treatment.However,most current works focus solely on the diagnosis of individual diseases,neglecting the interrelationships between diseases.Therefore,this study conducts research on both single-disease grading and multi-disease joint grading.For single-disease grading,this study takes Diabetes Retinopathy(DR)as the research object.The DR grading problem is modeled as a multiple instance learning problem,and a multi-instance grading network composed of multiple instance feature extraction modules(IFEMs)and a multi-instance grading module(MIGM)is proposed.To the best of our knowledge,this is the first work on DR grading based on multiple instance learning.Experimental results on the Messidor dataset and DDR dataset demonstrate significant advantages of the proposed multi-instance grading method compared to other methods.For multi-disease joint grading,this study focuses on DR and Diabetes Macular Edema(DME)as the research objects.The joint grading of DR and DME is modeled as a multi-task problem,and an Original Feature Fusion Network(OFFNet)based on the key-value query concept is proposed.The OFFNet consists of a specific feature extraction module(SFEM)and an original feature fusion module(OFFM).To the best of our knowledge,this is the first work on multi-disease joint grading based on the key-value query concept.Furthermore,the proposed OFFNet only requires training with image-level labels and excels at capturing long-range dependencies.Experimental results on the Messidor dataset and IDRiD dataset demonstrate significant advantages of the proposed OFFNet in terms of joint grading performance and capturing long-range dependencies compared to other methods.