| As the world population continues to grow,the incidence of skin cancer continues to rise.Early diagnosis and treatment can effectively reduce the mortality rate of skin cancer.Dermatologists can further screen for skin cancer through dermoscopy,but accurate diagnosis of skin lesions requires extensive experience,which requires a lot of time and money to train an excellent dermatologist.Also,misdiagnosis is common in skin cancer diagnoses.Therefore,there is a need for accurate and efficient methods for skin cancer diagnosis.Due to the continuous development of deep learning technology,computer-aided diagnosis has been widely used in the medical field.However,most scholars only use dermoscopic images to study the classification of skin cancer.Such single-modal data may not provide accurate lesion information,resulting in low classification accuracy.Meanwhile,since data of different modalities have different feature representations,how to effectively fuse multimodal data is a challenging problem.Based on this,this paper constructs a skin cancer classification model based on multimodal data fusion,realizes an auxiliary skin cancer diagnosis system based on deep learning,assists dermatologists in diagnosis,and improves diagnosis efficiency.The main work of this paper is as follows:(1)This paper proposes a skin cancer classification model based on multimodal data fusion.The model uses ResNet50 to extract dermoscopic image features,uses One-Hot encoding for patient metadata,and obtains metadata features through a multi-layer perceptron.In the feature fusion stage,this paper uses a variety of feature fusion methods,including simple splicing operations,bilinear pooling and attention mechanisms,and conducts seven-category experiments on the HAM10000 public data set.The experimental results show that the method using the attention mechanism has achieved the best classification effect,and the classification accuracy rate has reached 94.2%,which is a certain improvement compared with the common skin cancer classification model.(2)This paper designs and implements an auxiliary skin cancer diagnosis system based on deep learning.Starting from the actual needs of dermatologists,each functional module of the system was designed in detail,and the functions of the system were realized using development frameworks such as Spring Boot and Vue.At the same time,the Flask framework was used to deploy model services to assist doctors in skin cancer treatment.Diagnosis provides more accurate and efficient auxiliary diagnostic services for dermatologists,which has certain practical significance and application prospects. |