Research And Application Of Multimodal Brain MRI Image Segmentation Method Based On Transformer

Magnetic resonance imaging(MRI)image has become an important imaging method for brain disease detection and analysis due to its non-invasive and nonradiation damage.Brain tumor MRI image has become an important means for clinicians to diagnose and treat brain tumors.The segmentation of brain tumors needs to be completed by doctors with many years of experience.It is a highly professional work,and the segmentation task is very cumbersome and timeconsuming.With the increasing number of patients with brain tumors,in order to reduce the pressure of artificial segmentation,automatic segmentation of brain tumors based on computer-aided diagnosis has become a research hotspot.The traditional brain tumor segmentation methods,such as the method based on region growth and the method based on fuzzy clustering,have limited segmentation effects.In recent years,the brain tumor segmentation method based on deep learning has achieved good segmentation results.However,most models still have the following problems:(1)Due to its limitations,the models based on convolution neural network have not made better progress in the past two years,and the models combining convolution neural network and Transformer only use Transformer as a structure,which does not fully exert Transformer’s ability to model global information;(2)The model applied to other medical images was directly applied to the segmentation task of brain tumor,and there was no targeted design for the characteristics of brain tumor MRI images,resulting in the model failing to learn some important feature information;(3)The method used to solve the brain tumor MRI image segmentation task has too many parameters,too complex structure,high training cost and poor application prospects.Aiming at the problems of the existing models above,this paper proposes two brain tumor MRI image segmentation models Swin BTS and MSCAT U-Net based on Transformer and convolution neural networks.The main contents of this paper are as follows:(1)This paper proposes a brain tumor MRI image segmentation model Swin BTS based on Swin Transformer.Swin BTS is a novel 3D medical image segmentation model,which combines Swin Transformer,convolutional neural network and the U-shaped structure of coder-decoder,and defines the semantic segmentation of 3D brain tumor images as the sequence-to-sequence prediction task in this study.Swin Transformer proposes a kind of transformer similar to convolution shift window,which calculates and calculates self-attention in the window,thus greatly reducing the numuber of parameters.In order to extract context data,3D Swing Transformer is used as the encoder and decoder of the network,and convolution operation is used for up-sampling and down-sampling.The learning ability of simple Transformer’s detail feature information is not strong.This paper uses Hadamard product to design an enhanced Transformer module at the bottom of the network.The experimental results on Bra TS 2019,Bra TS 2020 and Bra TS 2021 data sets show that Swin BTS performs better than other segmentation models in brain tumor MRI image segmentation.(2)Based on Swin BTS and combined with the characteristics of multisegmentation of brain tumor MRI images,a unique brain tumor MRI image segmentation network MSCAT U-Net based on multi-scale transformer is proposed.Multi-scale Transformer is designed on the basis of PVT model.In the process of calculating self-attention,the self-attention layer can pay attention to the feature information of different scales by designing multi-scale query matrix and key matrix.In addition,multi-scale transformer uses space reduction operation,which greatly reduces the number of parameters.MSCAT U-Net adopts encoder-decoder structure,uses multi-scale transformer to extract shallow and deep spatial semantic information,and performs long-distance dependency modeling.MSCAT U-Net proposes a cross-attention module in the skip connection part to compensate for the semantic deviation between the encoder and decoder,and improves the ability of the model to extract the edge details.A large number of experimental results on three3 D medical image segmentation data sets(Bra TS 2019,Bra TS 2020,and BTCV)show that MSCAT U-Net has good segmentation performance in brain tumor and abdominal organ segmentation tasks.