Medical Image Analysis And Application Based On Multi-task Learning And Prior Guidance

In recent years,with the improvement of the digitization level of medical images and the rapid development of computer vision,medical image analysis technology has gradually become an important tool to assist clinical and is widely used in medical image science and pathological images.Medical image analysis can be viewed as a nonlinear mapping problem from image space to label space for different tasks.In recent years,solving this problem based on deep learning methods has received increasing attention,and a lot of related researches are still focusing on improving the performance of medical image analysis.Multi-task learning is the important method to solve the problem of nonlinear mapping from image space to multiple task label spaces.However,there may be a negative transfer phenomenon in multi-task learning.By incorporating prior knowledge,an explicit or implicit complementary relationship between different task label spaces can be established,thereby improving the stability and reliability of the algorithm.How to make full use of the relationship between multiple task label spaces and solve the problem of nonlinear mapping from image space to different task label spaces by incorporating prior knowledge guidance requires further thinking.To explore the above problems,this research uses multi-task learning with prior knowledge to solve the problem of nonlinear mapping from image space to different task label spaces.The research involves three levels of multi-task label spaces in medical image analysis:semantic segmentation(mask)space and heatmap localization(heatmap)space;bounding box regression(coordinate)space and classification(label)space;multi-domain image space.And according to the difficulties of the tissue samples analyzed from the macroscopic magnetic resonance image(MRI)to the microscopic cytology and histology image,the research guided by prior knowledge will be further carried out.By combining anatomical prior,semantic consistency prior,topological relation prior,and multi-domain style feature distribution prior,three prior knowledge-guided multi-task learning models for medical images are constructed.Specifically,it includes multi-task learning for segmentation,localization,and recognition of multiple instances with similar anatomical structures,multi-task learning for regression and classification of simple and difficult instances,and multi-task learning for multi-domain image generation and prediction of multiple unrelated instances.The main work involved in this research is summarized as follows:1.To solve the problem of nonlinear mapping from image space to mask space and heatmap space,a segmentation,localization,and recognition method guided by the fusion of anatomical prior and inter-task semantic consistency prior is proposed,and research is carried out on the spine MRI.This method first uses the sequential anatomical prior of the vertebrae to better learn the shared features.In addition,through the joint attention learning between the semantic segmentation and heat map positioning tasks,the segmentation attention features guided by the positioning task and the segmentation task-guided feature are obtained.Localization attention features are used to improve the results of the segmentation task and localization task respectively.At the same time,the semantic consistency prior is explicitly incorporated into the exclusive OR multi-task loss function,which not only provides a direct evaluation criterion for the semantic recognition relationship of similar vertebral centroid heatmap localization and semantic segmentation but also avoids the need for multiple subtasks.Weight adjustment of the loss function.The method was evaluated on a dataset of MRI(including T1 and T2 modalities)with different imaging fields of view.2.To solve the problem of nonlinear mapping from image space to coordinate space and label space,multi-task learning of rotating targets guided by topological relationship prior is proposed,and the research is carried out on the tall-cell subtype papillary thyroid cancer images.For difficult instances(tall cells)with ambiguous boundaries and different orientations and poses,the inter-class difference with easy instances(nuclei)is increased by weakly annotating the rotating rectangular box.Significantly improves classification performance while achieving tall cell length and width for visualizing difficult instances.In addition,an inter-class topological relationship learning for rotated instances is designed to indicate whether the regression results of difficult instances satisfy the topological relationship prior,constituting a dynamic topological relationship multi-task loss function.The method is compared and analyzed experimentally with horizontal target multi-task learning,rotating target single-task learning,and other rotating target multi-task learning methods.3.To solve the problem of non-linear mapping from image space to multi-domain image space,a multi-domain image translation model guided by the prior distribution of multi-domain style features is proposed,which is composed of hematoxylin-eosin staining(H&E)stained images simultaneously generate multiple functionally stained images and accurately predict multiple irrelevant positive signal instances.Compared with natural image generation diversity,histological image generation emphasizes learning domain-invariant content features and domain-specific style features.This method proposes to use Kullback-Leibler Divergence loss and histogram loss to constrain and separate the style feature spaces of different image domains respectively and obtain domain-specific style features.In addition,three functional staining images are generated at the same time,which can not only avoid the repetitive training of a one-to-one image generation network but also promote the multi-modal assisted diagnosis research that only provides a single staining.The method is evaluated on four datasets of lung lesions,lung lobes,breast,and atherosclerotic lesions.It involves generating nine kinds of virtual functional staining images from H&E and quantitatively analyzes the two tasks of image quality and positive signal prediction.