How Augmented Reality Will Make Surgery Safer (By Sarah Murthi, MD & Amitabh Varshney | Harvard Business Review)
Some of the biggest medical advances of the last few decades have been in diagnostic imaging—ultrasonogaphy, mammography, computerized tomography (CT), magnetic resonance imaging (MRI) and so on. The same forces that have propelled technology developments elsewhere—tiny cameras, smaller and faster processors, and real-time data streaming—have revolutionized how doctors use imaging in performing procedures. Almost every surgery involves some sort of a scan prior to incision. Even in emergencies, surgeons have ultrasound or CT to help guide the procedure. Imaging can now be performed in real time at the point-of-care during procedures, both big and small.
Yet, while imaging has radically evolved, how images are displayed is basically the same as it was in 1950. Visual data are always shown on a 2D flat screen, on displays that force health care providers to look away from the patient, and even away from their own hands while operating. Further, the images are not displayed from the perspective of the viewer, but rather from that of the imaging device: doctors have to use skill and imagination to understand and mentally project the images into the patient while they are doing procedures. Finally, different types of visual data are displayed separately, so doctors have to direct additional attention to mentally fusing multiple image types, such as angiography and CT, into a coherent representation of the patient. Acquiring this skill takes years of training.
Augmented reality (AR), a set of technologies that superimpose digital information on the physical world, has the potential to change all of this. In our research at the Maryland Blended Reality Center’s “Augmentarium,” we are prototyping AR applications in medicine, as are teams at Stanford, Duke and Johns Hopkins. In envisioned application, a surgeon using an AR headset such as Microsoft’s HoloLens would be able to see digital images and other data directly overlaid on her field of view. In such a scenario, the headset might display a hovering echocardiogram with vital signs and data on the characteristics of the patient’s aneurysm directly above the surgical field. The surgeon needn’t look away from the patient to multiple different displays to gather and interpret this information.
AR’s potential ability to concurrently display imaging data and other patient information could save lives and decrease medical errors. This is especially true for procedures done outside an operating room. The OR may be the safest place in the hospital, where one patient has an entire team of 4 to 8 dedicated doctors and nurses. Because everyone has pre-operative imaging, the procedures are generally well-planned. Anesthesiologists monitor the patient’s physiology and administer pain-controlling and life-saving medications. Surgical nurses make sure all of the necessary equipment is immediately available. Surgeons can be completely immersed in the operative task.