Retinopathy of Prematurity(ROP)is the first cause of blindness in preterm infants.If not diagnosed and treated in time,ROP can lead to lifelong blindness.Clinically,ROP is mainly diagnosed by professional ophthalmologists by interpreting the images collected by the widefield fundus imaging system(Retcam3),which inevitably leads to a certain degree of missed diagnosis and misdiagnosis.In recent years,artificial intelligence technology represented by deep learning has been widely used in the medical field and has achieved a series of results.The application of deep learning technology to assist ROP automatic staging is extremely valuable,but it still faces the following challenges:(1)As an important basis for clinical staging of ROP,some types of lesions have small distribution areas and insignificant characteristics,which makes it difficult to learn and represent the lesion knowledge and fail to be fully utilized;(2)The similarity of adjacent stages(especially stage 1 and stage 2,stage 2 and stage 3)is high and difficult to distinguish;(3)The cost of ROP image annotation is high,and there is no public image dataset for ROP staging and lesion segmentation.To address the above challenges,this paper proposes an end-to-end typical ROP lesion segmentation network framework and a ROP staging network framework based on semisupervised and contrastive learning by comprehensively applying deep neural networks,semisupervised learning,contrastive learning,and knowledge encoding techniques in the field of artificial intelligence.The main contributions include:(1)Aiming at the problem that the features of some lesions are not significant,and it is difficult to distinguish between different types of lesions and between lesions and surrounding areas,a ROP multi-lesion segmentation network framework KBSNet is proposed,which integrates knowledge coding and block-level balancing.With the help of image blocking and resampling technology,the importance of the image region containing lesion information in the network is strengthened through the block-level balancing strategy,and the learning ability of the network for insignificant lesions is improved.An image category coding based on domain knowledge is designed to guide the feature learning of the network,so as to obtain a more discriminative representation of different lesion categories and the background,and improve the ability of the network to distinguish different lesions.Experimental results based on ROPS350,DDR and REFUGE datasets show that KBSNet is superior to other reference models such as UNet,BPNet and STDCNet in terms of average AUC_ROC and average AUC_PR,with the highest values of 99.87% and 89.91% respectively;(2)A ROP image staging network framework based on semi-supervised and contrastive learning(SCSNet)is proposed to solve the problem of high similarity and difficulty to distinguish between adjacent stages of ROP images.Through semi-supervised learning,ROP lesion segmentation results are introduced into the staging network as auxiliary information to realize the supervision and guidance of ROP stage information.The supervised contrastive learning strategy is used to maximize the similarity of embedding vectors in the feature space of the same stage ROP images,minimize the similarity of embedding vectors in the feature space of different stage ROP images,improve the discrimination ability of the model for ROP images of different stages,and solve the problem that it is difficult to distinguish ROP images of adjacent stages.Experimental results based on ROP-C3922 and DDR datasets show that SCSNet is superior to the reference models such as Res Net,Dense Net and VGG in terms of Acc,quadratic weighted kappa,F1,Precision and Recall.The highest values were 95.80%,98.70%,94.46%,95.75% and 93.53%,respectively;(3)Aiming at the lack of ROP staging and lesion segmentation datasets,this paper constructed a ROP staging dataset ROP-C3922 and a ROP lesion segmentation dataset ROPS350. |