Research On Key Technologies Of Segmentation And Registration In Imaging Guidance

With the continuous innovation and development of medical imaging technologies and computer power,minimally invasive surgical intervention has been widespread used in clinic from the original trial to the clinical setting.New minimally invasive surgical interventions,which offer numerous interesting technical challenges,continue to demand new technological solutions.The technological components of tracking systems,segmentation,registration,visualization and software were involved in image-guided procedures.The trend toward image-guided interventions continues as increasing advances of the key technologies.During the intraoperative intervention,it is necessary that physicians precisely visualize and target the surgical site,which needs to integrate multimodal medical images for complementary information of anatomy and pathology.For instance,diagnosis and analysis of the spinal diseases requires CT and MR acquisitions in clinic.Some details of vertebrae are provided in CT image and a normal nature of soft tissues is identified in MRI,in which abnormal tissues can be easily and quickly found.Generally,2D X-ray imaging is used to be acquired during intervention,while CT and MR images are the preoperative modalities of choice,providing detailed 3D anatomical information that can be effectively exploited if they aligned to the intraoperative images.Registration,an essential component for all image-guided system,is the process of bringing two different coordinate system into spatial alignment.Commonly,preoperative images need to be registered with the patient on the procedure table by using the intraoperative images in the clinical scenario.Under this situation,a surgeon can observe the path line of the needle by employing a human visual system or virtual reality,which can improve the efficiency and affection of surgery,avoid some risks,shorten the operation time,and relieve patient suffering during surgery.After operation,the quantifying assessment of treatment frequently requires integration of multimodal images,such as anatomical imaging(X-ray,CT and MRI)and functional imaging(f MRI,PET and SPECT).Medical image analysis and processing,including image enhancement,segmentation,registration,and so on,can perform feature extraction and image fusion between different modal images,which can yield effective and reliable information for diagnosis,analysis and assessment of diseases in clinical practice.Typically,image-guided segmentation and registration have always been critical technologies in image-guided therapy system.In clinical applications,medical image segmentation and registration are mainly applied for diagnosis and analysis of diseases,preoperative planning,intraoperative guidance and quantitative assessment of postoperative efficacy.For example,in therapy planning system,tumor target delineation is very significant.It is no doubt that effective and accurate segmentation method improves efficiency,save time,perform radiotherapy in time to ease pain of patient,early treatment and early rehabilitation.Additionally,prior to treatment in radiotherapy,image registration is required to accurately positioning,and quantitative of assessment after radiotherapy.In this theme,medical image segmentation and registration technologies are investigating goals.On one hand,segmentation methods are proposed for extracting breast of ultrasound tomography image.On the other hand,according to the project acquirements,segmentation method based on deep learning is presented for segmenting vertebrae in CT and two registration algorithms are designed for multimodal alignment of CT and MR images and registration of 3D CT and 2D X-ray images,respectively.In short,this work mainly consists to several contents as following:(1)Efficient segmentation of a breast in ultrasound tomography using three-dimensional Grab Cut and its applicationUltrasound is widely applied for the screening of diseases and diagnosis of pathology,multimodal image-guidance and interventions with respect of the highlight efficiency,real-time,safety and low cost.As conventional 2D B-mode ultrasound images haven't the spatial information of anatomy,Neb et al invented 3D ultrasound tomography imaging technique to solve this problem.It can scan the entire breast using ring array transducers with B-mode,which reduces breast compression and human subjectively in image acquisition.As an emerging modality for whole breast imaging,ultrasound tomography offers many advantages in the screening and diagnosis of breast related diseases.Breast segmentation improves quantitative tissue analysis and other follow-up applications.However,ultrasound tomography has an inherent nature of conventional B-mode ultrasound.One of thresholding,region grow and cluster algorithms can't accurately extract breast region.Therefore,in the first stage,an interactive segmentation method based on 3D Grab Cut was designed for breast extraction in ultrasound tomography.Thirty reflection(B-mode)ultrasound tomography volumetric images were collected and segmented by the proposed method.The experimental results verified that the presented method had good performance and outperformed the other comparing methods.Next,an automatic segmentation method was designed.(2)Grab Cut-based automatic segmentation of ultrasound tomography image and its applicationBased on work(1),ultrasound tomography was still in testing stage in the clinical application.Image processing of ultrasound tomography was also in an exploratory period.To promote ultrasound tomography application in clinic,we deeply analyzed the essential characteristics of ultrasound tomography image.Then,we proposed an automatic segmentation method based on Grab Cut,combined with a series of image processing and computer graphics technology.Practically,we implemented automatically initial labeling for Grab Cut to achieve automatic segmentation.The experimental results demonstrated that the proposed method outperformed other comparing approaches in terms of efficiency and accuracy.Notably,with the improvement of ultrasound tomography device,the proposed method could extract mammary glands and segment nodules and tumors in ultrasound tomography image.(3)Spinal image segmentation method based on deep convolutional network and its applicationNowadays,deep learning is rapidly becoming the state of art,and widespread applied in the fields of medical image analysis.Deep learning has overcome problems and challenges encountered by traditional methods.In the literature,many merit deep convolutional neural networks have been designed for medical image segmentation and achieved promising results.More and more excellent networks will emerge in the future.Thus,we design a deep convolutional neural network based on U-net to segment the spine in CT.In training,we performed a cross process between training and testing and verified the model using images no involved in training and testing.On one hand,the experimental results demonstrated that the presented network model could effectively segment the spine.On the other hand,the network architecture needed to be further improved.(4)Intensity-based multimodal registration method of lumbar spine CT and MR images and its applicationIn the context of minimally invasive surgical interventions of multimodal image guidance spine,the diagnosis and treatment of spine-related diseases requires multimodal imaging acquisitions.Commonly,spinal CT and MR images are acquired for diagnosis and analysis of preoperative diseases,surgical planning,puncture path planning and simulation,and postoperative assessment of treatment efficacy.It is well known that CT yielded the most details of vertebrae,while MRI represented a very sensitive and accurate assessment of the lumbar spine anatomy,and detected abnormalities with soft tissues.In clinical application,multimodal registration is an essential component for combination of CT and MR images.In this work,we perform multimodal registration of CT and MR images in the lumbar region.The lumbar spine consists of vertebrae labeled L1 through L5.Typically,the lumbar spine is a remarked well-engineered structure of interconnecting bones,joints,nerves,ligaments and muscles all working together to provide support,strength and flexibility.The lumbar spine is a nonrigid tissue consisting of several rigid vertebrae.Additionally,CT and MR images acquired in clinic,a size of field of view between CT and MRI is same,but covering region in the lumbar spine is very different as it was selected by a doctor's individual experience.Obviously,a large number of multimodal registration approaches are not suitable for the lumbar spine.Thus,to solve this problem,we propose a multimodal registration method of three-dimensional CT and MR images of the lumbar spine.In this method,the multi-scale rigid registration applied for a global positioning,firstly.Secondly,the hierarchical deformation alignment is utilized for the local free-form deformation.Yet,it's usually argued that everything in image is handled with nonrigid object in the nonrigid registration algorithms,even though objects are obviously rigid structures or ones for which it is desired to reserve their rigidity.For Instance,there are vertebrae in the lumbar spine.Therefore,we introduce an bending energy penalty term in the deformable registration process for protecting the rigid bones from generating deformation.The experimental results express that the proposed method yields accurate and robust results for multimodal registration of CT and MR images in the lumbar region.The experimental results also demonstrat that this method is able to well implement the local deformation alignment while protecting the lumbar vertebrae from deforming.(5)3D-2D image registration method and its applicationBased on work(4),registration of pre-and intra-interventional data is one of the key technologies for minimally invasive surgery of the spine.Generally,3D-2D data registration method that utilizes 3D CT image as the preoperative data and 2D X-ray projection images as the intra-operative data.3D-2D image registration is an ill-posed problem,which can't be solved by conventional registration framework.To perform 3D-2D registration,the 3D and 2D data has to be brought into dimensional correspondence.Dimensional correspondence can be achieved by transforming the 3D data into 2D or by transforming 2D data into 3D.While the former method leads to 2D-2D registration,the latter approach leads to 3D-3D registration.More specifically,dimensional correspondence can be achieved either by the projection,back-projection,or reconstruction strategy.Nowadays,mostly 3D-2D image registration,utilizing various initial positioning devices,is often difficult to reproduce and implement in the current clinical setting.However,3D-2D image registration technique has very importantly clinical significance.The scope applications of 3D-2D registration in medical interventions is fairly broad,for example in image-guided surgical intervention and radiotherapy.Therefore,we proposed a 3D-2D registration algorithm based X-ray source initial location and multi-object optimization and distribution strategy.Global optimization pattern search algorithm has been employed in 3D-2D registration,and a divide-and-conquer registration process is utilized to better avoid the local minima.The proposed method has two novel points.Firstly,the initial positioning of the X-ray source uses the off-line DRR image set,which makes the algorithm more flexibility.Secondly,a multi-object mesh adaptive direction search method is used to implement a coarse-to-fine registration process utilizing a step by step strategy.Additionally,we employ two CIRS phantoms of head and abdomen to test the proposed method.3D CT images were acquired by using the clinical CT scanner,and 2D X-ray images were shot by using a CBCT platform constructed in our laboratory.Finally,the experimental results verified that the proposed method can solve the out-of-plane problem in optimization search as well as restrain the local minima further.In order to make the algorithm meet the clinical requirement,it is necessary to further reduce the time consuming.