Deep Learning-Based 3D/2D Registration For Spine Surgery Navigation

Due to the existence of important anatomical structures such as nerve roots and blood vessels,compared with the direct visual feedback of traditional open surgery,minimally invasive spinal surgery based on indirect image feedback puts forward higher requirements for positioning.The surgical navigation system realizes intraoperative guidance through the registration of the preoperative image coordinate system and the intraoperative reference coordinate system,and is expected to better guide minimally invasive spinal surgery.This paper studied the key technology—3D/2D registration of preoperative 3D CT images and intraoperative 2D X-ray projection images.In order to build dimensional correspondence between the 3D/2D data to be registered,most methods use projection strategy to convert the problem into 2D/2D registration.However,the projection causes the loss of spatial information of the 3D anatomical structure in the image,resulting small capture range and timeconsuming iterative process in the subsequent registration.Reconstruction can essentially convert the problem to 3D/3D registration,effectively integrate the information of multi-view X-ray images,extends the capture range.On this basis,this paper studied the reconstruction-based 3D/2D registration method,the specific works are as follows:1)Segmentation-guided Orthogonal-view X-ray Image Registration with CT Image:In order to alleviate the loss of spatial information by the forward projection strategy,we proposed to reconstruct the 3D representation from the orthogonal-view X-ray projection images,and then perform 3D/3D registration in the manifold space.The 3D expression reconstruction is guided by 3D vertebrae segmentation,focusing on the inherent rigid anatomical structure,and providing more significant and precise shape and position information for registration.By sharing multi-scale features between segmentation network and pose estimation module,dimensional correspondence and pose estimation are integrated into a unified end-to-end framework,which facilitates mutual supervision and improvement.The geodesic distance loss is used to adapt to the parameter space of 3D poses and further improve the robustness of the model.Experiments prove that the reconstruction-based strategy can make full use of 3D spatial information and alleviate the ambiguity in 3D/2D registration.The registration speed meets real-time requirements and can further advance spinal surgery navigation.2)Geometric Model Integrated Deep Learning Method for Synthesis of Novel View X-ray Projection Images:The acquisition of multi-view X-ray projection images will increase the radiation dose and complicate the clinical workflow.We study a strategy for obtaining novel view projections from a given view projection,as a complement and extension to 1)when only single-view projections are available.Assuming that projections from different views share common texture features,while their geometric features are view-specific,feature disentanglement is performed on projected images to better model the relationship between projections of different views.Then,using the physical model of X-ray imaging,the 2D geometric features are backprojected into the 3D space,and then re-projected according to the target view.Finally,the target view projection image is generated by combining the geometric features of the target view and the shared texture features.Integrating the geometric relationship of the physical world into the deep learning model constrains the consistency of geometric information in 3D space and increases the interpretability of the model.Experiments show that this method can well model the geometric relationship between the source and target views and reflect the geometric structure in the projected image.It establishes the basis for subsequent reconstruction-based 3D/2D registration when only single-view X-ray images are available.