| PurposeTo construct a bimodal image artificial intelligence model based on color fundus photographs and OCT images that can automatically diagnose pathologic myopia,grade myopic maculopathy based on ATN classification system,screen for patients requiring referral,determine the treatment options and assess the disease progression.Methods656 patients with myopia,and 1184 groups of same-machine color fundus photographs with macula-centered OCT images were included.Pathological myopia was defined and graded based on the ATN classification system.3 trained annotators formed the annotation team,and each image was independently annotated by at least 2 ophthalmologists,with annotation including lesion,ATN classification,referral,and possible treatment options.After annotation,the training,test and validation sets were stratified and randomly divided into roughly 60%,20%and 20%.This study used the convolution module of ResNet50 to construct a multiple-modal multiple-instance learning model.Accuracies,quadratic-weighted κcoefficients were used to evaluate the model’s ATN classification results;an area under the receiver operating characteristic curve(AUC),sensitivity,specificity and F1-measure were used to evaluate the model’s ability to diagnose pathological myopia,to screen patients who may need to be referred and to determine the treatment options that patients need.ResultsThe model which we constructed had an AUC of 0.962,an accuracy of 0.9583,a sensitivity of 0.9466,and a specificity of 1.0000 for diagnosing pathological myopia.The accuracy of the model was 0.7500,0.7875,and 0.8583 for the A,T,and N classifications,respectively,with quadratic-weighted κ coefficients of 0.8566,0.7823,and 0.5831,respectively.In identifying patients who required referral,the model had an AUC of 0.968 and an accuracy of 0.8958,and in determining the different treatments of patients,the model had an accuracy between 0.8417 and 0.9750 and an AUC between 0.925 and 0.976.As a comparison,we also trained AI models based on unimodal images of color fundus photographs.The unimodal models had accuracies of 0.7292,0.6000,and 0.8250 on A,T,and N classifications,respectively,with quadratic-weighted κcoefficients of 0.8188,0.2736,and 0.6098,and the accuracy of the unimodal models in determining the patient’ s treatment ranged from 0.7125 to 0.9333,and the AUC ranged from 0.658 to 0.824.Compared with the unimodal model,the bimodal model has more significant advantages in ATN classification and determining patient treatment options.ConclusionThe AI model based on color fundus photographs and OCT images can accurately achieve automatic classification of pathologic myopia,diagnose pathological myopia,screen patients who need to be referred and determine the next treatment plan for patients.Compared with traditional AI models based on unimodal images of fundus photographs,this model can provide a more comprehensive assessment of the patient’s pathology and can be better applied for myopia prevention and control. |