3D Reconstruction From Endoscopic Video Images

The application of endoscope in minimally invasive surgery will help to guide future development of surgery. However, when using endoscope to exam internal cavities inside the body, the physicians could only see the two-dimensional (2D) endoscopic images, the three-dimensional (3D) information inside the body and the real location of the endoscope are usually not available under the monocular endoscopic surgery. In order to solve these problems, this thesis introduces a method of 3D reconstruction from monocular endoscope video images, which can track the trajectory of the endoscope and reconstruct the 3D scenes of the internal cavities. The 3D reconstruction method includes following procedures:1. We calibrate the endoscope using nonlinear camera model and correct the distortion of the endoscopic images;2. We discuss the KLT algorithm in tracking the endoscopic image features and indicate that the KLT algorithm would fail when the scale and/or the brightness of the image change suddenly. We propose an improved feature tracking algorithm which use SIFT descriptor and feature motion estimation between the two consecutive video frames. We also propose a simple and convenient outlier detection strategy based on motion consistency constraints.3. After detecting and matching features in the endoscopic images, we use the multiple-view matrix based iterative decomposition algorithm and the triangulation method to reconstruct 3D structure from motion. From this, we get the 3D projective reconstruction results.4. Because the physicians may change the endoscope focus during the surgery, we use endoscope self-calibration method to computer the focus length and update the 3D projective reconstruction results to 3D Euclidean reconstruction results;We evaluate our method using the nasal cavity endoscopic videos acquired from Shanghai 6th Hospital and successfully track the trajectory of the endoscope and reconstruct the 3D scenes of the nasal cavity. These experiments prove that our implemented method is feasible for 3D reconstruction from monocular endoscope videos.