Classification Of Pneumonia Images Based On Transformer Hybrid Model

As a serious respiratory disease,pneumonia can not only cause fever,but also threaten the life and health of people.However,in recent years,the research on image classification of pneumonia becomes a hot topic due to the globally spread of COVID-19.In post-epidemic era,COVID-19 and other influenza viruses will still invade the human’s body.Therefore,early diagnosis and treatment are extremely critical.In recent years,with the progress of artificial intelligence technology and the development of deep learning,more and more research on medical image has been carried out.Training and analyze disease datasets through deep learning methods and development of specialized auxiliary diagnosis systems have gradually become the mainstream direction of medical image research.In this context,based on X-ray and CT images of pneumonia patients,this paper adopts deep learning and transfer learning methods to propose two improved models based on Transformer network structure to realize three classifications of two kinds of pneumonia datasets.The specific work of the paper includes:(1)Classification tasks based on X-ray images of pneumonia.The Swin T model was improved,and the pneumonia X-ray image classification model based on Swin T-CNN was proposed.This model and four other transfer learning models were used for training on the data set of 36249 lung X-ray images successively.Multiple preprocessing operations were carried out on the data set in the early stage of model training to further expand the number of images.At the same time,each model can better fit the data to achieve the best training effect.The experimental results show that the improved Swin-CNN hybrid model has stronger generalization ability.The experimental results show that in the classification task of lung X-ray images,the model reaches the highest accuracy of 98.5%,which is better than other transfer learning models and 2.6% higher than Swin T model.(2)Classification tasks based on CT images of pneumonia.The VIT model was improved and the residuals in the Res Net50 model were added.The pneumonia CT image classification model based on VIT-RE50 was proposed and compared with other classical transfer learning models.In this experiment,a large scale CT data set was used,and the total number of images reached 425024.In the experiment,the VIT-Re50 model was normalized by GN layer to further accelerate the convergence rate of the model.The experimental results show that the prediction accuracy,recall rate and F1 score of VIT-Re50 on CT image data are 94.3%,93.7% and 93.4%.(3)This paper also studied the visualization of the lesion area in CT images of patients with pneumonia,and compared the lesion detection effects of different models with Grad-CAM technology.Experimental results showed that this technology could effectively generate the thermal map of the lesion area,so as to achieve the effect of assisting the diagnosis of patients with pneumonia.