Research On Deep Learning Based Non-Rigid Multi-Modal Medical Image Registration

The multi-modal medical image registration is a research hotspot in medical image processing and it is of great significance to develop highly efficient and accurate image registration methods.The multi-modal medical image registration is important for such clinical fields as surgical navigation,functional positioning,and disease diagnosis.However,due to the existence of the nonlinear intensity difference,the complex non-rigid deformations,the possible lack of feature correspondence between multi-modal medical images,the traditional methods and the existing deep learning-based registration methods are difficult to achieve accurate and highly efficient registration.The research of more accurate and robust registration algorithms based on the structural representation,hierarchical supervised registration and unsupervised registration can be explored to solve the above-mentioned problems.1)Aiming at the nonlinear intensity difference between multi-modal medical images,a structural representation method based on the feature fused-Principal Component Analysis Network(FPCANet)is proposed for multi-modal medical image registration.In the proposed method,the FPCANet is built by cascading principal component analysis and trainned on numerous medical images firstly.Then,the reference and float images are processed by the trainned FPCANet and the features extracted by various layers in the FPCANet are fused to construct the structural representations of float and reference images rerspectively.Experimental results based on Atlas,Brain Web,and RIRE datasets show that compared with the traditional registration methods,the proposed method can provide higher registration accuracy and better robustness.2)Aiming at the time-consuming problem caused by the iteration in traditional image registration methods,and the drawback of restoring the deformation of edge and smooth regions indiscriminately in the existing deep learning methods,the two-stage generative adversarial networks(TGAN)based supervised registration method is proposed.In the proposed method,the medical image registration problem is decomposed into two sub-problems: the deformation restoration and and the intensity estimation.The first stage GAN restores the image deformation by estimating the structural representation of the registered image indirectly;the second stage GAN generates the final registered image based on the results of the first stage.Experimental results based on Atlas,Brain Web,RIRE datasets show that compared with the existing methods,the proposed method can provide higher registration accuracy and better robustness.Especially,the registration efficiency of the proposed method is higher than that of the traditional methods.3)Aiming at the need of labels in the supervised registration and the deformation field folding problem in the unidirectional registration,the GAN combined with the free-form deformation(FFD)and symmetric constraint(FSGAN)based unsupervised registration method is proposed.The FSGAN consists of one generative network(G)and two discriminative networks(D).The two D networks are used to discriminate whether the bilateral registration is completed respectively.The FSGAN uses the FFD to reduce the number of prediction parameters,and introduces the symmetry constraints into the bidirectional registration,which can avoid the change of image topology during registration.The experiment is conducted based on Brain Web,IXI,HGG and LPBA40 datasets and the results show that the FSGAN can avoid the folding of deformation fields in the unidirectional registration effectively,and provides higher registration accuracy than the existing unsupervised registration methods.4)Aiming at the possible lack of anatomical structure correspondence between multi-modal medical images,the registration methods based on simultaneous segmentation and registration networks(SSRNet)is proposed.The network contains two sub-networks: segmentation network(SegNet)and registration network(RegNet).The SegNet realizes the semi-supervised learning by combining the unsupervised loss based on the level set function and the supervised loss based on the mean square error,which can segment the target area of the reference and float images effectively under the premise of reducing the dependence on the label.The RegNet uses the segmentation results of the SegNet to achieve the unsupervised registration.The SegNet and RegNet adopt an alternant training strategy,which can reduce the adverse effect of segmentation errors on the subsequent registration process effectively in the segmentation-based-registration method.The experimental results based on the PROMISE12 and beagle dog kidney datasets show that compared with the existing methods,the simultaneous segmentation and registration method can reduce the adverse effects of image nonlinear image intensity differences and other interference on the registration process effectively,which can better correct the deformation of soft tissue,and provide an effective new method for such applications as soft tissue puncture surgery navigation.Aiming at the challenges in the multi-modal image registration,such registration methods as based on structural representation,supervised learning,and unsupervised learning are proposed to achieve accurate and highly efficient registration,which can promote the wide application of the multi-modal medical image registration to the aided diagnosis and treatment effectively.