| Lupus erythematosus is a chronic,recurrent autoimmune skin disease that not only endangers life and health but also causes fear and discrimination from others.In the initial stage of the disease course,the skin and mucous membranes are mainly damaged,which leads to skin lesions.Once it develops to the advanced stage,various organs of the patient will be damaged,organs will fail,and the mortality rate will be greatly increased.Therefore,early screening and diagnosis of lupus erythematosus are crucial.At present,the diagnosis of disease mainly relies on dermatologists to complete the diagnosis through imaging examinations.However,the skin lesions of the disease are variable and heterogeneous.The same subtype will have a variety of different skin manifestations,and different subtypes will have highly similar skin lesions.Moreover,considering only a single modal data results in limited information.This clinical diagnosis method is highly uncertain and susceptible to subjective factors.Therefore,it is a major trend in the future to help dermatologists to make an objective and efficient diagnosis of lupus erythematosus with the help of computer-aided diagnosis technology and multi-modal data.The target size of multicolor immunofluorescence images is different,and the position information and texture information of the same subtype are complex and changeable.A lesion area in a clinical lesion image may be a mixture of lesions of different subtypes.The semantic correlation between different modal data cannot be fully captured in multimodal fusion.These problems make the auxiliary diagnosis of lupus erythematosus an extremely challenging task.Focusing on these difficulties and challenges,this paper has carried out a series of research works using the latest deep learning technology.The main contents are as follows:1)This paper proposes an auxiliary diagnosis algorithm for lupus erythematosus based on a deep fusion of multimodal data.The algorithm extracts the corresponding deep feature expressions according to the inherent characteristics of each modal data and at the same time uses a feature fusion device to fuse the features of different levels of each modality,and finally provides subtype classification for lupus erythematosus disease.The algorithm mainly includes Feature Aggregation Module(FAM),Label Enhancement Module(LEM),Cross-modal Feature Fusion Module(CMFM).FAM mainly solves the difficulty of complex and changeable position information and texture information in multi-color immunofluorescence images.It models the local spatial relationship of different image patches at different scales and captures deeper global texture information between image patches at the same time.For clinical skin lesions images,LEM replaces the original one-hot label distribution by finding a similar distribution of labels that is more in line with the actual data,thereby improving the model’s ability to distinguish samples with confusing or noisy labels.CMFM further excavates the deeper inter-correlation between modalities by considering the correlation between modalities and the independence within modalities,and it realizes the deep fusion of modal features.2)In this paper,the lupus erythematosus multimodal data set is collected from 25 tertiary hospitals across the country with a total of 446 cases,and each case ensures that the information on each modality is complete.All data are labels collected after codiagnosis by at least two experienced dermatologists.3)This paper conducts extensive experiments on clinical datasets to verify the effectiveness and generalization ability of the algorithm.In addition,to compare the accuracy of the algorithms more intuitively,this paper also tests the actual diagnostic ability of the model through man-machine comparison.Finally,the quantitative results and visual effects show that the algorithm proposed in this paper achieves the best diagnostic performance,which provides a new research idea for the auxiliary diagnosis of lupus erythematosus. |
PDF Full Text Request