Augmented Reality Navigation in Robot-Assisted Surgery

In the last few decades, major complications in surgery have emerged as a significant public health issue, and so the practical implementation of safety measures to prevent injuries and deaths in different phases of surgery is required. Augmented reality (AR) is considered one of the most promising solutions for safer procedures in several surgical specialties. Fusing patient-specific preoperative information, typically 3D models extracted from CT scans or MRI, with real-time surgical images allows the surgeon to have detailed information on the anatomical structure of the surgical target intraoperatively. The coupling of AR and robotics represents the next step toward introducing awareness into the surgical room, thus enhancing the surgeon’s perceptual, cognitive, and manipulative capabilities. This chapter describes the main areas involved in an AR navigation system integrated into a robotic teleoperated platform. It will describe the modalities to obtain a patient-specific virtual model, the methodologies to develop an AR navigation system, and the methods to implement it in a teleoperated surgical robotic platform. Recent advances in the field are also presented, providing as an example a novel integrated system for real-time AR navigation in robotic minimally invasive surgery (RMIS), composed of a robotic endoscopic camera, teleoperated implementing a software-based remote center of motion (RCM), and an AR navigation software based on an initial manual registration of virtual 3D models with the real anatomy.