Multi-modal Medical Image Generation And Segmentation Algorithm Based On Attention Mechanism And Generative Adversarial Network

In recent years,medical image processing methods based on deep neural networks have made significant progress in lesion segmentation,disease progression tracking,and assisting treatment in brain disease imaging.However,the existing deep learning methods still have certain limitations in assisting lesion segmentation.Firstly,limited training data can easily lead to overfitting.Paired data is crucial for improving the performance of segmentation networks,but obtaining a large amount of highquality paired annotated medical images is very difficult.Secondly,using imbalanced data can cause instability in the training process of segmentation networks,especially for complex lesion edges and small lesion areas,which make network training difficult.Finally,without data augmentation and other techniques to optimize the structure of imbalanced training data,the fitting ability of deep neural networks will be limited.This thesis focuses on brain tumor and stroke Magnetic Resonance Imaging(MRI)as research objects and proposes two improved algorithms based on CycleGAN(Cycle-consistent Generative Adversarial Networks,CycleGAN)for generating missing modality MR images.Meanwhile,the generated MR images are validated using the segmentation networks 3D UNet and Swin-UNet on existing publicly available datasets.The main research work of this thesis is as follows:(1)Introducing a multi-modal medical image generation network called CASPGAN(Coordinate and Spatial Attention based Generative Adversarial Networks)based on coordination and spatial attention.First,the CBAM(Convolutional Block Attention Module)module is inserted into the downsampling and backbone networks of CycleGAN,and the impact of spatial attention and channel attention on the generation network is evaluated.Finally,the spatial attention with the best comprehensive performance is selected.Then,coordinated attention is introduced to replace the residual network in CycleGAN,which further enhances the information interaction between the spatial width and height dimensions while reducing the network parameters.The algorithm is applied to the BraTS2020 dataset,and the experiment shows that:(a)CASP-GAN has significant improvements in training and prediction speed.The generator operation time is reduced to one-third of the original algorithm,and the parameter quantity is reduced from 7.83 M to 0.8M.On the RTX2060 platform,the generation time for a single patient modality image is reduced from 11.5 seconds to 0.67 seconds,allowing the network to run on lowerperformance devices.(b)Compared with the original CycleGAN network,CASPGAN’s four generation indicators exceed the original algorithm in most cases or remain at the same level.(c)By generating Flair modality images using T1,the generated images are used to train the U-Net segmentation network and segment lesion areas.Compared with using T1 and T2 modality images for segmentation alone,all four segmentation indicators are significantly improved.(2)Introducing CHSE-GAN(Channel Attention and Squeeze-Excitation based Generative Adversarial Networks),a multimodal medical image generation network based on compression-excited channel attention.To address the issues of the original CycleGAN and the proposed CASP-GAN in generating brain stroke images with multiple regions,complex boundaries,non-uniform appearance,and poor image details,we incorporate a compression-excited module and channel attention based on batch normalization.This module redistributes the weight information among different channels to enhance the generation capability of edge details.The algorithm is applied to the ISLES2015 dataset,and the experiments show the following results:(a)CHSE-GAN achieves superior generation performance in two tasks:generating Flair images from T1 and generating T2 images from T1,surpassing both CASP-GAN and the original CycleGAN in Chapter 3 with only a small increase in parameters.For the task of generating Flair images from T1,PSNR is improved by8% and RMSE is improved by 18.2%.(b)Compared to using T1 modality images alone for segmentation,employing the generated Flair and T2 modality images through T1 and utilizing the generated multimodal images to assist in training the Swin-UNet segmentation network for lesion area segmentation significantly improves precision without compromising the other three evaluation metrics.It achieves a 4.8%improvement over the original algorithm.In summary,This thesis proposes CASP-GAN and CHSE-GAN,two generative networks,to address the difficulty in obtaining paired data and segmentation of multimodal brain tumor and stroke images.These networks are used to generate missing modality paired MR images.The proposed method is validated on the BraTS2020 and ISLES2015 datasets,and it shows improvement in both the speed and quality of the generated images.The generated images can be used to improve the effectiveness of brain lesion segmentation,and therefore,the proposed generative algorithm has a good clinical disease-assisting diagnostic effect.