Research On Semantic Modeling And Retrieval Of Medical Images Based On Hybrid Bayesian Networks

The research of medical image semantic retrieval is a new hotspot in the field of medical image retrieval, and should be settled urgently in medicine, too. It is an interdisciplinary study on medical image understanding, denoting the synergy of medicine, image processing, pattern recognition, computer vision, machine learning, database and artificial intelligence. Semantic modeling and semantic similarity measure are its difficulty and key. The fundamental task of image semantic modeling is to extract implicit unknown high-level semantics in order to make up"semantic gap". The dissertation tries to study medical image semantic modeling approach based on hybrid Bayesian networks (BN), thought about characters of medical images and demand in medicine. It includes the study of multi-level semantic statistical models of medical images, capture of object semantics and high-level semantics, and semantic similarity measure. To validate these methods, they are applied in the prediction of astrocytoma malignant degree, and semantic models of astrocytoma malignant degree and a semantic retrieval system are designed.The main contributions of the dissertation are as follows:1. It proposes a medical image semantic modeling method using hybrid BN embedding Conditional Gaussian (CG) models.(1) Depending on the characters of medical images and advantages of Bayesian networks, it is proposed to use Bayesian networks to model medical image semantics. But classical Bayesian networks are adapt to discrete variables alone, and automated image features often are continuous. To use continuous ones in classical Bayesian networks, hybrid BN embedding CG is proposed. The CG makes a fuzzy discretization for continuous image features with probabilistic and uncertain nature. A semantic model with low-level image features alone is built, called BN-CG-Low. The experiment results show that this model can extract semantics from low-level image features, with good description of images.(2) Based on BN-CG-Low, semantic model BN-CG fusing low-level image features and middle-level semantics is designed, to improve semantic precision and recall and describe image content better. It has better precision and recall, comparing with BN-CG-Low.2. A three-level semantic model integrating Gaussian mixture models (GMM) into hybrid BN, named BN-GMM, is given.In medical diagnosis, doctors give a conclusion after thinking over pathological objects from different views and on different levels, so a three- level medical images semantic model is proposed, in which GMM is used to capture middle-level object semantics, and then embedded into a Bayesian network. The experiment between BN-GMM and BN-KNN demonstrates that BN-GMM achieves better precision and recall, with better interpretation.3. Hierarchical semantic similarity measure, based on distance in semantic probability space, is proposed.Different level semantics has different weights in the semantic models, and semantic probability reflects semantic brief. It accords with the habit in medical diagnosis. Therefore, a hierarchical semantic similarity measure is studied, in which, semantic similarity measure depends on distance in its posterior probability space on every layer. The method is applied in semantic retrieval of astrocytoma malignant degree, and gets satisfying query performance.4. A three-level semantic model integrating SVM into hybrid BN, named BN-SVM, is designed.Lots of research proved SVM can get better precision than GMM, in case of small samples. So a three level medical images semantic model is designed, in which SVM are used to extract middle-level object semantics, and then embedded into a Bayesian network. From experiments between semantic model BN-SVM, BN-KNN-Low with KNN instead of SVM, BN-GMM-Low with GMM instead of SVM and BN-CG-Low with CG instead of SVM, BN-SVM gets better performance.In case of the same samples in this paper, our new methods outperform the BN with KNN as object semantic detectors, when they are used in modeling medical image semantics.The research of hierarchical knowledge expression, semantic extraction and semantic similarity measure provides a solution to enable medical image semantic retrieval by using variables keywords at different semantic levels.