Study On Key Issues Of Image-Guided Minimally Invasive Treatment For Liver Cancer

Liver cancer is a key disease that threatens the life and health of the Chinese people.Surgical resection or liver transplantation are the preferred treatments for liver cancer,while ablation and chemoembolization are effective interventional treatments.With the rapid popularization of laparoscopy,laparoscopic liver resection has attracted more and more attention due to its advantages of less trauma and shorter postoperative recovery time,especially in recent years,three-dimensional(3D)laparoscopy has become a popular choice for surgeons providing depth perception during operations.However,it is still difficult for surgeons to locate the anatomical structures such as vessels and lesions of intrahepatic under 3D laparoscopy,and it is also difficult to integrate the preoperative surgical planning into the intraoperative well.For the clinical demands of 3D laparoscopic liver resection and percutaneous puncture navigation,the main research contents and achievements of this dissertation are as follows:(1)Guided by the specific clinical needs of liver resection and ablation,an augmented reality-assisted navigation system for 3D laparoscopic hepatectomy and a percutaneous puncture navigation system are introduced in this dissertation.Preoperative models are fused with intraoperative 3D laparoscopic video in former system,which expands the surgeons' field of vision and facilitates them to explore the location of intrahepatic vessels and tumors.A pointcloud registration algorithm is embedded into the latter system to achieve marker-free registration.A variety of experiments have been conducted for verification,including ex vivo,in vivo,phantoms and animal experiments.Among these,the navigation accuracy of augmented realityassisted navigation system in ex vivo and in vivo experiments is 6.04±1.85 mm and8.73±2.43 mm,respectively.The error for the proposed puncture navigation system in phantom and in vivo experiments are 2.76 mm and 3.02 mm.The effectiveness and accuracy of the systems have been verified by experiments,as well as the feasibility of clinical application.(2)Focusing on the key technologies of developing an augmented reality-assisted liver resection navigation system,a progressive hand-eye calibration algorithm based on a single invariant point is proposed to obtain the hand-eye transformation in 3D laparoscopic augmented reality navigation.The invariant point is imaged by 3D laparoscope,then the 2D coordinate of the invariant point is calculated and map it to the 3D coordinate system of the camera.At the same time,a tool fixed at the distal end of the 3D laparoscope is captured by an optical positional system,and then the handeye transformation is estimated based on the Levenberg-Marquardt optimization algorithm according to the image coordinates of the invariant point,as well as the position and pose the distal tool.Experiments show that the re-projection error of the proposed algorithm under the optical positional system can reach 0.71 mm to 1.32 mm.(3)Estimating the depth of the surgical scene based on stereo laparoscopic image is one of the key steps for developing an augmented reality navigation system.The result of the depth estimation will be used to register with the preoperative model.To solve the problem of difficulty in obtaining the groundtruth depth data in training a learning based method,a depth prediction method based on deep learning network fusing with traditional stereo knowledge is developed in this dissertation.First,the disparity image is obtained by using traditional stereo computer vision method,and then the unreliable disparity assignments are eliminated by using confidence measures to generate a proxy ground truth disparity image,which is used to guide the training of the depth estimation model.Moreover,a smoothing loss term is employed to constrain adjacent pixels with similar appearance to generate smooth surface.The predicted error of this algorithm on public datasets is 1.84±0.40 mm and 1.49±0.41 mm,respectively.(4)An unsupervised depth estimation network for stereo laparoscopic image is proposed to address the issue of the accuracy degradation of the depth prediction model under imperfect camera parameters.By introducing a vertical correction module of disparity,the matching caused by imperfect camera calibration parameters is adjusted to ensure the matched points are located on the same horizontal scanline.Also,the generative adversarial network and residual mask are embedded into the depth estimation network to improve the accuracy of imperfect rectified stereo images.The mean absolute error of the proposed model on the public dataset is 3.05 mm.