| In recent years,Augmented Reality（AR）guided thermal ablation（TA）has attracted extensive attention in the treatment of liver tumors.Compared with traditional 2D computed tomography（CT）image-guided methods,AR technology can provide accurate 3D image information of lesions and can effectively reduce the operation difficulties caused by "heads up" effect.However,current AR navigation systems for real-time reflection of intra-operative organs and lesions are mostly composed of preoperative models,while existing technologies only consider the corresponding relationship between surface markers and tumors.Therefore,existing AR technology is still difficult to be directly applied in real-time surgical scenarios.In this thesis,a liver tumor ablation method syncretizing digital twin and holographic augmented reality navigation is proposed,which can be used for liver tumor elimination under free breathing or other surgical environments.The main contents and achievements of this thesis are as follows:（1）A two-step external/internal correlation model was proposed to implement real-time intra-operative liver surface prediction based on generalized moving least squares（GMLS）with reference liver surface as input,and the association model between body surface markers and liver surface was constructed.The location of intra-operative liver surface was predicted by GMLS,which updated the location of surface markers caused by free breathing movement in real time.Considering the error caused by the surface roughness predicted by GMLS,Laplacian smoothing was introduced to smooth the predicted results to achieve accurate prediction of intra-operative boundary conditions.A convolutional neural network（CNN）was designed for the respiratory prediction of the liver internal fatal heterogeneous structures.With the movement to get real-time intra-operative hepatic surface,CNN is introduced for non-rigid registration of preoperative surgery for predicting the displacement field of the displacement field is applied to the reference of the liver and tumor blood vessels can get the real-time prediction of location.（2）We design a cross-platform holographic augmented reality（AR）surgery navigation system to restore good in liver surface and its internal tumor blood vessels and holographic three-dimensional information superimposed on the real operation situations,by putting forward a method of the actual registration of the virtual model and real object registration together,finally achieve the goal of three-dimensional navigation surgery.（3）A dynamic adaptive method was proposed to realize the dynamic time prediction of intra-operative body surface markers,so as to compensate the time delay caused by the computational complexity of association model and cross-system data transmission,and to fulfill the requirements of real-time surgical operation.In this thesis,we reconstructed liver 4D-CT data by manual segmentation,and conducted an animal experiment to verify the proposed method,the iodine was used as the liver tumor.The results demonstrate the effectiveness of the proposed method,and the proposed method can also be used in other surgical scenarios. |
