Application Of Reinforcement Learning In Localization And Registration Of Liver Interventional Surgery

Liver cancer is one of the cancers with a very high mortality rate,and its first choice of treatment is surgical removal of the tumor.However,patients with liver cancer are often accompanied by liver cirrhosis or are already in the middle and advanced stages of liver cancer at the time of diagnosis.About 80% of patients cannot get the opportunity of surgical resection.In recent years,interventional therapy with the characteristics of less impact on liver function,less trauma,and definite curative effect has gradually become the main treatment plan to deal with the above situation.Interventional treatment of liver cancer refers to the introduction of surgical instruments/drugs into the lesion area for local treatment under the guidance of medical imaging equipment such as CT,MRI,or ultrasound.Therefore,multi-modal image fusion technology is the core part of interventional therapy for liver cancer and the key to ensure the success of the treatment,which determines the positioning accuracy of surgical instruments in interventional therapy.However,the widely used dual-modality image fusion of preoperative CT and intraoperative ultrasound is plagued by problems such as image registration and liver deformation,which limit the further clinical application of interventional therapy for liver cancer.Based on the prior knowledge of liver anatomy,this paper decomposes the image fusion problem in interventional surgery into two parts: the positioning of the key blood vessels in the liver and the multi-modal medical image registration.The work content is listed as follows:1)First of all,this article selects a stable structure,the bifurcation of the left and right branches of the liver portal vein,in the rich vascular system of the liver as the target point.For this purpose,we build a single-agent deep reinforcement learning(DRL)model to locate this point.The research is based on the two public liver 3DCT data sets,the MICCAI 2017 Liver Tumor Segmentation(LITS07)data set and the 3D-DIRCADb data set.The results show that DQN,Double DQN and Dueling DQN and other DRL models detect the left and right branch points of the portal vein.The average distance error is between 3.31 and 3.85 mm,which proves the performance of the DRL model in locating the characteristic points of anatomical structures in the complex 3DCT images of the liver.2)Secondly,this paper attempts to build a multi-agent reinforcement learning(MARL)model under the Dueling DQN algorithm to locate the target plane composed of three blood vessel bifurcation points on the relatively stable anatomical structure of the sagittal part of the left hepatic portal vein.The result shows that the accuracy of the Dueling DQN and DQN models on 32 cases of data from the Zhongda Hospital affiliated of Southeast University is between 3.57-3.94 mm,Dueling DQN models have weak advantages in all three target points.The comparison of accuracy is carried out between single-agent and MARL models.The above two public data sets are used to test the different performance of the MARL model and the single-agent model under the Dueling DQN and DQN algorithm in locating the three vessel bifurcation points on the para-sagittal part of the left portal vein.The experimental results show that the MARL model shows the advantages of 0.14-2.19 mm at the three landmarks,and the training time of the model is about 30% shorter than that of the single-agent models.3)In order to explore and verify the image fusion registration of 3DCT and 2D ultrasound based on the prior knowledge of liver anatomy,this paper uses the position information of the three vessel bifurcations obtained by the MARL model to slice the corresponding 2DCT image.Then the correlation between the physical value of CT and the acoustic impedance coefficient of human tissue is constructed,and the corresponding simulated ultrasound image is obtained based on the Rayleigh scattering theorem and the acoustic wave transmission model.On the other hand,a two-dimensional MARL model is built to locate the three branches of the para-sagittal part of the left portal vein in real ultrasound images.Finally,according to the results of RL models,the regions of interest selected from real and simulated ultrasound images are framed and their similarity is calculated.The experimental results on the CT and ultrasound data of 5 cases of the same patient have achieved good agreement,which proves that the use of prior knowledge of liver anatomy and DRL models can help the initial fusion registration of CT and ultrasound images in the interventional treatment of liver cancer.