The Potential of AR in Preoperative Oncovascular Surgical Planning

Today, Prof. Emanuela Marcelli leads this lab at the Hospital IRCCS Azienda Ospedaliero-Universitaria di Bologna. The lab was started with the mission to use advanced 3D technologies — like virtual reality (VR), augmented reality, simulation, 3D modeling, and additive manufacturing — to provide medical professionals with new tools for preoperative planning, intraoperative guidance, and personalized surgical tool and implant design.

It began about seven years ago, initially for the team to gain experience with segmentation software. Gradually, they acquired 3D printing equipment and AR/VR visualization systems to expand their capabilities. “We started by responding to requests from a few surgical specialties — cranio-maxillofacial, vascular, urology — until today, where we have grown to cover 16 hospital operating units,” explains Laura.

“Our strengths in building the lab were a strategic location, a multidisciplinary team, and the right technology. Our daily work includes ensuring lab sustainability, quantifying the benefits of 3D technologies, tracking costs, following Medical Device Regulation, and promoting our activities.”

Recently, the team at the lab initiated a study on AR to explore its advantages in preoperative planning for oncovascular surgery, a treatment often applied to tumors involving or arising from major vascular structures. They expect many potential benefits, including the ability to isolate specific structures within the anatomy and view both the CT scan and 3D model at once.

“The goal of this study is to see how following the preoperative plan in AR impacts the effectiveness of the procedures,” explains Laura. “We’re just at the beginning with one case enrolled, but by the end, we’ll have compared 20 cases planned conventionally using bi-dimensional CT scans versus 20 cases planned using 3D technologies, i.e., a virtual 3D model projected on a normal 2D display, a 3D-printed model, and a virtual model displayed in augmented reality.”

The technical team at the eDIMES lab will use Mimics to prepare the 3D anatomical models. They’ll access the reconstructed 3D models and prepare them for surgery on Microsoft HoloLens using the AR functionality in Mimics Viewer. The process is largely similar to preparing a 3D-printed model — they segment the anatomy from patient imaging, prepare the virtual model, and then upload the model to the AR app. From there, surgeons can interact with their patient’s anatomy in a more immersive and intuitive way. In addition to all of these benefits, it’s often more affordable than 3D printing models.

“We’ve received various requests from oncovascular surgeons for help with complex cases in the abdomen. 3D printing can be especially expensive for these applications because surgeons want larger, multi-color models to view all the anatomy in detail. Using AR instead could offer a cheaper, more comprehensive alternative that allows them to visualize the anatomy in different variations, for example, with or without the arteries or veins,” says Laura.

Within the study, Laura and the other researchers will compare three variables that can impact the patient outcome: how much blood is lost through hemorrhagic events, how completely the surgeon removes the tumoral mass, and how closely the surgeons are able to follow the preoperative plan. She hypothesizes that AR will contribute to an improvement in each of these areas.

“I believe AR is poised to make significant advancements in healthcare over the next five years, especially in surgical assistance, medical training and education, and hospital workflow optimization,” says Laura.

The first advancement Laura foresees is in surgical assistance. She believes AR could improve precision and outcomes by providing enhanced visualization of critical information — such as patient vitals and anatomical landmarks — superimposed onto the surgical field. “Moreover, AR could allow surgery to become increasingly minimally invasive by displaying the location of instruments and anatomical targets inside the body, thus reducing the need for large incisions,” she says.

When it comes to medical training and education using AR, Laura foresees immersive learning experiences for students and professionals, made possible by superimposing anatomical information over a real body or 3D-printed model, allowing for realistic simulations in a risk-free environment.

In fact, she plans to test it out more with otolaryngology residents during a summer school program. There, they’ll explore the use of AR for Mimics Viewer in real pathological cases collected from surgeons and reconstructed in 3D. Future research will also give them the opportunity to test other AR headsets, such as Magic Leap 2.

It’s a great opportunity for residents to get experience with the technology and for Laura to observe how it aids in the teaching process: “It will be interesting to try to quantify the improvement in the educational process.”

And, last but not least, improving workflows. Laura predicts that hospitals will be able to streamline their workflows by using AR to guide staff through procedures, reducing errors and improving healthcare efficiency. Keep an eye out for these developments!