Multimodal Fusion For MR Brain Tumor Image Segmentation Algorithm

Medical imaging technology is essential to assist pathological diagnosis and disease analysis.MR images segmentation has gradually become a popular research direction in the field of image segmentation.We face many difficulties when useing MR image,for instance complex multi-modal data fusion,single feature information,and fuzzy partition edge.The common solutions mainly use convolutional neural networks to build deep networks for image segmentation.However,when convolutional neural networks process MR images,there are some problems such as difficulty in capturing global information and excessive model parameters.Glioma brain tumor is set as the research object.This paper intends to build a new algorithm for the fusion of attention mechanisms and convolutional neural networks,as well as a double decoder branch edge segmentation algorithm to solve these problems.The main research contents and innovations of this paper are as follows:(1)Aiming at the integration of local and global attention mechanisms,a new segmentation algorithm CSNet with an "encoder-decoder" structure is designed.The encoder is a parallel feature extraction branch consisting of two branches based on CNN and Transformer where features of the same size are fused.The decoder consists of two Swin Transformers,and two learnable parameters are introduced for feature lift sampling.The multiscale feature maps from the encoder are merged by skipping connections and same-size features in the decoder.(2)Aiming at the difficult situation of fuzzy segmentation of edge details,the Edge Seg algorithm of the double decoder branch is designed.The algorithm uses the improved Res Net framework to build an encoder,respectively designing two different decoding channels for the edge and the main part,and fusing the two branch feature maps before outputting the final segmentation results.In this algorithm,the attention-free mechanism is introduced to improve the ability of global information extraction without increasing the amount of system computation.Finally,a three-dimensional Canny operator is introduced to generate an edge mask to strengthen the edge branch segmentation ability.In the ET,TC,and WT regions of the Bra TS dataset,the Dice of the CSNet were 81.88%,88.57%,and 89.27%,and those of the Edge Seg algorithm were 83.16%,88.79%,and 90.31%.Experimental results show that the two frameworks designed in this paper are superior to similar segmentation algorithms for MR Glioma images.