| Gastric cancer is a malignant tumor of the digestive system with high incidence and mortality rate.Due to insufficient screening of early lesions and its insidious nature,most patients are diagnosed with advanced gastric cancer.A lot of studies have found that early diagnosis of gastric cancer is crucial to the treatment and prognosis,and how to improve the detection rate of early gastric cancer is an urgent problem in the medical field.As the "gold standard" of gastric cancer diagnosis,pathological diagnosis mainly consists of tedious microscopic examinations of pathological section by experienced pathologists.Due to the different subjective experiences of pathologists,there is a lack of consistency in the diagnosis results.With the development of computer technology,artificial intelligence pathology diagnosis can provide pathologists with some diagnostic references and it can transform qualitative analysis into quantitative analysis.Most of traditional artificial intelligence pathology diagnoses are based on color images,and the limited information of color images restricts the improvement of diagnosis accuracy.With the introduction of hyperspectral imaging technology into the medical field,hyperspectral images containing spatial and spectral information are able to capture the differences in the spectral domain among tissues and provide more information.This paper establishes a pathological dataset of gastric cancer precancerous lesions containing hyperspectral images and RGB images based on microscopic hyperspectral imaging technology and investigates the identification methods with deep learning.This paper proposes a two-stage auxiliary information for pathological diagnosis by achieving the initial screening of disease through classification task and precise localization of the lesions through segmentation task.Deep learning requires a large amount of data to enhance the robustness,however,it costs a lot to acquire the medical data and annotations.To solve such problems,a selfsupervised architecture based on 3D spatial-spectral transformation is designed to provide effective pre-trained models for downstream tasks.It is shown that the complementary information between different imaging modalities and joint training can improve the accuracy of these tasks.Therefore,this paper designs dual modalities feature fusion networks.In the classification task,this paper builds feature exchange and fusion modules between two Res Net18 networks by using skip connections,and rescales the feature maps of two branches in channel dimension by SE modules.In order to enhance the commonality between two branches and learn more robust features,this paper constrains the training between two branches using Pearson distance.In the segmentation task,this paper designs a Cross Attention learning strategy to facilitate the communication between features in two U-Nets and uses an agent to reduce the computational effort.The calculation amount is reduced while the accuracy is guaranteed.In addition,a distillation loss is used to align the prediction results of two branches and improve the generalization of the network.Experimental results show that the pre-trained models designed in this paper can effectively improve the accuracy of downstream tasks and improve the accuracy and Kappa coefficient by about 7.10% and 10.72%.Most importantly,the pre-trained models can save about 50% of the annotation workload for the classification task.Besides,the accuracy of the initial disease screening can reach 96.15%,and the accuracy and Dice of the second stage of segmentation can reach 96.53% and 91.62%,which proves that the joint training of hyperspectral images and RGB images can effectively improve the accuracy of auxiliary diagnosis.In conclusion,the dataset established in this paper can provide data support for subsequent studies,and the proposed methods based on feature fusion of two imaging modalities can provide a new idea for the screening of gastric cancer precancerous lesion and precise location of lesions. |
PDF Full Text Request