Methodology Research Of Myocardium Ultrasound Image Segmentation Based On Graph Cut And Deep Learning

Myocardial infarction(MI)is a severe cardio-vascular disease that threatens human health,leading to the subsequent death of cardiomyocytes and vascular cells in the vicinity site of the infarction.Ultrasound cardiogram is the only technique used to observe dynamic images of the heart and provides a non-invasive way to diagnose and monitor heart conditions.However,some inherent drawbacks of ultrasound(US)imaging,such as low contrast,speckle noise,signal dropout,acoustic shadow,cause the myocardial tissue indistinguishable from the background.And myocardial segmentation and assessment are the keys in the morphology and function study of myocardium.Therefore,it is valuable and challenging to investigate accurate and effective segmentation algorithms of myocardium ultrasound(MUS)images.The common methods of medical US image segmentation can be divided into seven classes:thresholding-based,clustering-based,watershed-based,graph-based,active contour model based,Markov random field based and neural network based segmentation methods.Where neural network or deep learning has recently become a hot research topic.Some researchers investigated the semantic segmentation algorithms.Others investigated the traditional machine learning methods combined with prior information and intuitive feeling.In order to extract the whole myocardial tissue well,this paper studies the continuous graph cut method based on the feature combination of superpixels and neighborhood patches and the deep learning segmentation method based on VGG16-UNet,respectively.The continuous graph cut method based on the feature combination of superpixels and neighborhood patches(fast Superpixels and Neighborhood Patches based Continuous Min-Cut,fSP-CMC)consists of five parts:(1)graph construction of graph cut model,(2)feature extraction of superpixels and neighborhood patches,(3)definition of novel similarity measure,(4)setting of interactive labels,and(5)solution of continuous graph cut model.The US image is constructed by a graph,which depends on the features of superpixels and neighborhood patches.A novel similarity measure is defined to capture and enhance the features correlation by combination of Pearson correlation coefficient and Pearson distance.And the interactive labels provided by user play a subsidiary role in the semi-supervised segmentation to compensate for the lack of US image quality and improve the accuracy of segmentation.Then the continuous graph cut model is solved via a fast minimization algorithm based on augmented Lagrangian and operator splitting.The comparative experimental results with the continuous graph cut method based on the features of neighborhood patch(fast Neighborhood Patches based Continuous Min-Cut,fP-CMC)show that the Dice,Precision,and Sensitivity values offSP-CMC were basically higher than those of fP-CMC,which indicates that the segmentation results of fSP-CMC is better than the segmentation results of fP-CMC.The deep learning segmentation method based on VGG16-UNet migrates the learned feature representation in the VGG16 model pre-trained on ImageNet into the U-Net model,then fine-tunes and trains on our own training set.Since the pre-trained model VGG16 is a three-channel classification model,it is necessary to change the number of input channels to 1,and remove the fully connected layers,then use it to replace the encoder part of U-Net for migrating the features.The decoder part of U-Net uses the deconvolution layer to upsample the feature map and restore it to the original size layer by layer,for achieving the image pixelwise classification,i.e.,the segmentation of a single image.In the upsampling process,the fusion of high-level features and low-level features is performed layer by layer by skip connection to further improve the accuracy of image segmentation.The training data is too small and thus augmented 10 times online.The model is evaluated using 10-fold cross-validation method.The comparative experimental results with the U-Net show that the Dice,Precision,and Sensitivity values of VGG16-UNet were basically higher than those of U-Net,which indicates that the segmentation results of VGGI6-UNet is better than the segmentation results of U-Net.However,compared with the segmentation results of fSP-CMC,the segmentation accuracy and robustness of VGG16-UNet were still not high enough.It is linked to the number of training data that is still not enough to capture the common US image artifacts.Meanwhile,it is also related to the segmentation network.There is still much room for improvement to adapt to the inherent drawbacks of the US image.In conclusion,both methods have achieved good segmentation results.The segmentation accuracy for the MUS image by fSP-CMC is higher,but fSP-CMC is still limited in static segmentation.And VGG16-UNet can meet the dynamic segmentation of ultrasound cardiogram,but the segmentation accuracy and robustness are still not high enough.In the future,one of the research plans is to combine the deep learning method with the fSP-CMC method,for approximately estimating two boundaries as interactive labels near the initially rough contour segmented by the deep learning method,then using the fSP-CMC to optimize,which is expected to transform the fSP-CMC into an automatic segmentation method.The segmentation step of the deep learning method still plays a key role,another research plan in the future is to improve the network architecture of the segmentation model and further improve the segmentation accuracy and robustness for MUS image while increasing the number of the training data.