Image guided liver surgery using augmented reality

Radiological images are not available real time for interaction during most surgical procedures. Clinically, this makes operations on non-palpable and non-visible liver masses difficult without intra-operative imaging such as ultrasound. While ultrasound will help identify masses and surrounding vital structures, it is dynamic and operator dependent. Three-dimensional, cross-sectional imaging is more reliable and reproducible for intra-operative navigation and pre-operative planning for these types of masses, but is not easily integrated into the operative field. This work is a first step towards integrating medical images into an Augmented Reality system which will allow rendering of 3D images directly on to the diseased organs or surgical location of interest, thus allowing for intra-operative interaction. Clinically, this will be important for open or minimally invasive surgery of liver lesions that are not visible or palpable for both intra-operative navigation and pre-operative planning.; A prototype system was developed that consists of an inanimate liver model, a head mounted monoscopic display and a position sensor. A 3D liver was reconstructed from two-dimensional CT scan images. The segmentation problem is addressed first and both automatic segmentation and semi-automatic segmentation are studied. Once the images are segmented, the 3D model is generated by triangulation and then displayed in a virtual environment. Finally, the virtual liver model is placed into a simulated workspace in synchrony with a physical model of a liver prototype to test the algorithms and concepts. The work is implemented in Visual C++ on an Intel PC.