Real-time Registration Of 2D/3D Lung Images Based On Spatio-temporal Information Fusion

Image-guided radiotherapy(IGRT)is a common method for treating early-stage lung cancer,which aims to achieve precise treatment by dynamically adjusting surgical instruments based on real-time acquisition of the patient’s anatomical structure during the surgery.However,due to the high radiation dose of the 3D Cone Beam CT(CBCT)technique for acquiring patient anatomy,which is prone to secondary injury to patients.Therefore,the method of using low-dose 2D X-ray images to establish the correspondence with 3D CT images for determining the patient’s anatomical structure,known as 2D/3D image registration,has become a hot research topic.Addressing the time-consuming and unstable issues of traditional 2D/3D lung image registration methods,this paper proposes a deep learning-based approach to achieve a more precise and stable registration process by effectively utilizing the feature information in 2D X-ray images.Additionally,a 2D/3D lung image registration system is designed to achieve real-time registration transformation.The main contributions of this paper are as follows:(1)Proposed 2D/3D lung image registration method based on dual attention mechanism residual network.In this paper,we establish a deep learning-based 2D/3D registration framework using Principal Component Analysis(PCA),and propose a residual network based on a dual attention mechanism(DA-Res Net)to achieve the 2D/3D registration process.This network can effectively extract motion information from the image,achieving a more accurate prediction process.Experimental results show that DA-Res Net has good predictive ability(MAE: 14.41 ± 8.85,R2: 0.95 ± 0.07)and reconstruction results(NCC: 0.9981 ±0.0006,SSIM: 0.98 ± 0.04).(2)Proposed 2D/3D lung image registration method based on spatiotemporal networks.This paper extends the deep learning-based 2D/3D registration framework to a temporal input approach and applies for the first time the spatio-temporal fusion network in 2D/3D registration research.In addition,the paper proposes both the spatio-temporal concatenation fusion network(CNN+LSTM)and spatio-temporal parallel fusion network(Conv LSTM).Experimental results show that the parallel fusion approach outperforms the serial fusion approach,and the Conv LSTM performs well in both model prediction capability(MAE:47.15 ± 9.2,R2: 0.92 ± 0.10)and reconstruction results(NCC: 0.9967 ± 0.0010,SSIM: 0.95± 0.03).(3)Designed and implemented real-time 2D/3D lung image registration system.In this paper,a real-time 2D/3D lung image registration system is constructed by combining the spatial network DA-Res Net and the spatiotemporal network Conv LSTM.The system layout and logic processing are implemented through page development and model deployment,and the model inference stage is optimized by structural reparameterization to achieve realtime operation.After testing,the system can achieve real-time registration of 2D/3D lung images under single and multiple inputs.