Adaptive Patch And Multiscale Learning Based Approaches For Super Resolution Of 4D-CT Lung Data

Lung four-dimensional computed tomography (4D-CT) is of great values in the application of tumor localization and individualized precise radiotherapy, which is accomplished by sorting the images into multiple CT volumes corresponding to various states of the respiratory cycle. The characterization of respiratory motion provided by 4D-CT data sets can be used to determine the extent of tumor motion for margin selection in the treatment planning process, which is crucial for high-precision radiation therapy. However, due to the inherent high-dose exposure associated with CT, dense sampling along the superior-inferior direction is often not practical, resulting in much lower inter-slice resolution when compared with the in plane resolution and the anisotropy of data. Therefore, to obtain the correctly proportional image, coronal and sagittal image needs to be interpolated along the superior-inferior direction according to the ratio between in-plane resolution and slice-select resolution. Nearest-neighbor interpolation and linear interpolation are commonly used, but these simple approaches do not introduce new information and recover the high frequency components, resulting in blurring image.The main objective of this paper is to enhance the axial resolution of lung 4D-CT data. Super resolution (SR) is a commonly used technique in enhancing image resolution. The basic principle behind SR is to combine non-redundant information in multiple low resolution (LR) frames to generate high resolution (HR) images. Hence, based on the characteristic of lung 4D-CT data, we propose two kinds of SR reconstruction algorithms.Firstly, we develop an adaptive patch-based POCS approach for super resolution reconstruction of 4D-CT lung data. The main work of this approach is as follows:(1) Lung 4D-CT data provides respiratory-synchronized LR image sequences of the lung. Thus, we regarded LR images of different phases at the corresponding position as different "frames". We can then employ model-based SR approach to achieve an HR image. Each SR algorithm relies on an acquisition model. SR reconstruction always inverts the degradation model without loss of generality to reconstruct an HR image from undersampled, noisy, and blurred LR images. A typical observation model is related to the original HR image with observed LR images.(2) When observed lung 4D-CT data, we find that for corresponding images of different phases, the local structures sometimes differ because of respiratory artifacts. To eliminate the interferences or artifacts found in reconstructed HR images, we first perform in raster-scan order in the image to sample the patches with pixel overlap, from left to right and top to bottom. For a LR patch, to determine patches that can be adopted to reconstruct HR patch, we search for patches with similar structures in the corresponding spatial region in all other phases. Specifically, patch similarity with respect to a candidate patch is evaluated based on the Euclidean distance. For better characterization of structural patterns, we use both intensity and derived feature to represent each patch. Based on the distance measurement, we set a threshold for adaptive selection of similar patches. We can evaluate the effectiveness of each candidate patch by calculating the distance between candidate patch and LR patch. The candidate patch with a distance lower than threshold will be selected. The result shows that with patch adaptive selection, dissimilar structures can be filtered for SR reconstruction, which cannot be easily achieved through global SR reconstruction.(3) Motion Estimation. As lung motion is not a rigid action, we therefore employed deformable registration approach to estimate motion fields. We adopted the Demons algorithm in lung 4D-CT data to estimate the motion field between the patches. The method is relatively simple, high precise and high speed, without any pre-processing of the image sequence.(4) Based on the motion field, we employ POCS method to reconstruct HR patches. POCS method is the classical algorithm of reconstruction based SR technology, which is simple and allows the convenient inclusion of a priori information that is always desirable and often even crucial in ill-posed problem. Finally, assemble all HR patches to output the HR lung 4D-CT image. Global consistency is ensured by enforcing a global reconstruction constraint to optimize output image. The experimental results demonstrate that the proposed method can reduce artifacts effectively and yield clearer lung 4D-CT images with enhanced edges and details.Secondly, we propose another learning based SR technology to improve the resolution of lung 4D-CT data.In general, the registration or block matching is utilized in model-based SR reconstruction method for motion estimation, which restricts the efficiency and accuracy of reconstructed results. However, learning based SR approach can avoid this process and has become the active area of SR technology. The basic principle behind learning based approach is to help recover details in LR images by utilizing the relationship between the HR and LR images, learned via training images. Therefore, this kind of method needs HR images and corresponding LR images simultaneously to build the training data. While the HR coronal and sagittal image is absent caused by low axial resolution of 4D-CT data. So, what kind of data can be used to make up HR and LR patch pairs for training the two coupled dictionaries is the first challenge in this proposed method.Furthermore, the objective of learning based SR technology is a series of overlapping patches, which is general based on the same size. However, the anatomical structures are manifested in different scales in lung 4D-CT data. Hence, how to choose a patch size that can capture anatomical information in multiple scale simultaneously is the second challenge in this method.Aiming to the above difficulties we develop a self-similarity-based multiscale sparse representation SR approach to enhance the resolution of lung 4D-CT image. The proposed approach not only addresses the two problems effectively, but also generates favorable result both qualitatively and quantitatively. The main content of this approach is as follows:(1) We use the assumption of cross-scale self-similarity to solve the first problem, which is based on the observation that small-scale structures tend to redundantly repeat themselves throughout an image. This concept has been widely employed in image processing. We exploit the cross-scale self-similarity between transverse image and coronal and sagittal images in different patch sizes (16x16,8x8,4x4, respectively). And structure similarity (SSIM) is adopted to measure the similarity between patches. The effectiveness of cross-scale self-similarity has been proved through experiments. Therefore, based on this assumption, we utilize the transverse HR and LR image patches as training data for learning the two coupled dictionaries.(2) After having training data, sparse representation approach is adopted to reconstruct HR patches. There are two dictionaries DH and DL trained to have the same sparse representation for each HR and LR image patch pair. The principle of sparse representation SR is to recover HR patch from DH by directly using the sparse representation of corresponding LR patch in terms of DL. Finally, assemble all HR patches to output the complete HR image.(3) The solution to the second problem is to introduce the concept of multiscale which can avoid the difficulty of selecting a patch size in advance, especially for the lung data with the existence of multiscale features. Multiscale analysis employed in image processing can be traced back to 1980s, and numerous algorithms has been presented to design the best multiscale dictionary. Instead of designing the predefined dictionary for image reconstruction, learning multiscale dictionary has attached wide attention recently. In current study, we adopt a quad-tree based strategy to partition the overlapping image patches and build joint global dictionary composed of different sized blocks. The result demonstrates that the multiscale process can capture more anatomical information and generate image with significantly enhanced structures.Data used in this paper were obtained from the DIR-lab. The dataset was launched in 2009 by the University of Texas M.D. Anderson Cancer Center (Houston, TX). The dataset consists of ten cases of 4D-CT. Each case was acquired by employing a GE medical system (General Electric Discovery DT PET/CT scanner). For each case, the 4D-CT images covered the entire thorax and upper abdomen, and contained 10 phases, including the extreme inhale and exhale phases. The two algorithms are estimated by simulated and real images, respectively.(i)Simulated data:In this data set we obtained only HR transverse images. Therefore, HR transverse images were utilized to simulate LR images based on the observation model. We use simulated images not only for exploring their capability to reconstruct HR images of lung 4D-CT images, but also for testing the influence of several critical parameters in this paper.(ii)Real data:The coronal and sagittal LR images from different cases are adopted to reconstruct HR images and evaluated visually.The experimental results illustrate that compared with global POCS method, our patch-based algorithm can reduce artifacts effectively and generate more precise HR lung 4D-CT image. This approach also yields clearer images with enhanced edges and details than cubic spline interpolation and BP algorithm. Compared with single scale based algorithm, our self-similarity-based multiscale sparse representation method can capture more anatomical information and generate image with significantly enhanced structures. Compared with bilinear interpolation, this method displays quality improvement in terms of resolution. And compared with POCS algorithm, it can avoid the process of registration which restricts the efficiency and accuracy of reconstructed result. Moreover, the quantitative evaluation of this two proposed algorithms are also better than other methods.