Research And Application Of Image Segmentation Model Based On Machine Learning And Multi-modal Fusion

With the development of the application of big data and artificial intelligence technology,image segmentation has become the main direction of computer vision research.And it has been successfully applied in many fields such as medicine,and security.A series of image segmentation algorithms have been developed based on machine learning,and great progress and breakthrough success have been achieved in theory and application.However,traditional image segmentation methods focus on analyzing single-modal images.Due to the limited single-modal information,the accuracy of segmentation is limited,and information fusion in a multi-scene environment cannot be achieved,which limits the application of image segmentation.In this study,the subjects are infrared thermal imaging and magnetic resonance imaging(MRI).In this paper,three improved multi-modal fusion image segmentation algorithms are proposed,starting from applied research and algorithm research,and taking advantage of multi-modal fusion and machine learning algorithms in feature expression.The algorithm is applied to the actual collected data and the existing public data set to obtain good experimental results.The main research contents of this article are as follows:(1)An infrared thermal image solar panel region segmentation method based on multi-modal feature fusion is proposed to achieve accurate segmentation of the region of interest of infrared thermal image.Aiming at the problem of mis-segmentation caused by complex infrared thermal image interference,the algorithm first extracts contrast,entropy and gradient feature maps.And then constructs multi-modal feature fusion maps.Finally,the region filling is carried out to segment the region of interest.The algorithm is applied to the infrared thermal images of photovoltaic solar panels actually collected.The results show that the proposed algorithm has higher precision(0.9306),higher recall(0.9028).F1 index(0.9144)and.J index(0.8515)are better than other algorithms.And it has less manual labeling,can be effectively applied to the segmentation of infrared thermal image.(2)An automatic MRI brain tumor segmentation algorithm based on 3D ResU-Net and multi-modal fusion is proposed.Aiming at the problems of multi-region,blurred edges and difficult segmentation of brain tumors,the algorithm makes full use of the MRI modal differences and U-Net multi-scale advantages.First,multi-layer convolution is used to extract hierarchical features.Then,feature stitching and fusion are performed,and residual network is added to prevent the gradient from disappearing.Finally,the segmentation mask is reconstructed to realize the segmentation of the brain tumor.Apply algorithms to public data sets.The results show that the Dice coefficients(whole tumor,core and enhanced areas:0.8882,0.8667,0.6958)and Hausdorff distances(10.8776,13.9730,and 9.907)of the algorithm are superior to other algorithms.And it takes less time on GPU,which is beneficial to clinical application.(3)A multi-modal image generation and brain tumor segmentation algorithm based on CycleGAN-MRI is proposed.To solve the problem of lack of medical image data and time-consuming and laborious multi-modal image acquisition,the algorithm firstly uses CycleGAN-MRI unsupervised learning,and uses the generator and discriminator to continuously "game" to generate the optimal multi-modal image.Then,the fusion of the original modal and the generated modal are used to realize the tumor segmentation.Apply the algorithm to public data sets for performance evaluation.The results show that the algorithm's MAE(T1?T2,T2?T1:2.886,2.401),PSNR(25.302,24.461)are superior to other algorithms.And it can accurately achieve brain tumor segmentation.The algorithm has great application value.In conclusion,this paper proposes three improved multi-modal fusion image segmentation methods for infrared thermal images and MRI,which can be applied to infrared thermal image segmentation and medical auxiliary diagnosis and treatment systems.This paper has completed a series of theoretical and applied research on image segmentation.