Exploiting shared image structure fusion in multi-modality data inversion for atherosclerotic plaque characterization

In many subsurface sensing problems, single-sensor information quality is poor, due to factors such as constrained sensing geometries and limited energy penetration. In such cases, there is interest in combining information from multiple complementary sensing modalities to enhance signal estimation. Our work focuses on fusing heterogeneous subsurface reconstructive imaging modalities by extending variational boundary-preserving image smoothing methods from the computer vision community to multisensor environments involving physics-based signal processing.; Specifically, we develop a unified joint optimization framework for fusing and estimating boundary structure shared among multiple imaging modalities; we simultaneously exploit these fused boundaries to enhance image reconstructions. Our approach incorporates observation models for each modality's data, alignment parameters for registering the modalities, and models relating the common boundaries to the modalities' reconstructions. We carefully choose our optimization criteria to reduce the impact of ill-posed or badly conditioned observation kernels and to obtain more stable, reliable, and robust reconstruction estimates in the face of observation, modeling, and computational error sources.; The specific application that motivates this work is the imaging of vulnerable atherosclerotic plaques. Vulnerable atherosclerotic plaques are blood vessel wall segments, damaged by inflammation, that are prone to rupture, releasing clotting agents that can cause heart attacks or strokes. No single imaging modality has yet demonstrated the ability to detect these vulnerable lesions reliably. We demonstrate our approach by fusing edge observations from Magnetic Resonance (MR) and Computed Tomography (CT) blood vessel imagery into a single, shared, underlying tissue boundary field estimate, while we simultaneously exploit the fused boundaries to enhance and reinforce edges in the estimated MR and CT tissue reconstructions. We demonstrate our approach by fusing boundary field estimates from MR and CT blood vessel imagery into a single estimated underlying tissue boundary field, while simultaneously estimating the original imagery to better estimate tissue characteristics and structure.; More generally, we present an approach for multi-modality subsurface data inversion and fusion based on shared image structure. This approach allows for better estimates of the characteristics and structure of the underlying scene. These objective criteria form a unified optimal multi-modality boundary-preserving inversion and registration framework.