Towards a Clinical Stereoscopic Augmented Reality System for Laparoscopic Surgery

We describe our development of a complete real-time stereoscopic augmented reality system that overlays laparoscopic ultrasound (LUS) images on stereoscopic laparoscopic video for conventional laparoscopic surgery. The system was designed and developed to achieve near-term clinical evaluation as a primary goal. Special consideration was paid to system interactivity, accuracy and easy integration within the existing clinical workflow. Custom-designed fixtures for the two imaging devices were created to avoid their recalibration in the operating room and thus to minimize setup time. The system was assembled on a rolling cart to make it portable for the use in the operating room. Utilizing our optimized design and hardware-accelerated implementation, the system achieved a low system latency of approximately 150 ms. The LUS image-tovideo registration accuracy, measured in terms of target registration accuracy and recorded separately for the left and right eye channels of the stereoscopic camera, was 2.38±1.15 mm and 3.66±1.42 mm.