Embracing A New Disruptive Technology: 5 Reasons Why Hospitals Can't Afford to Ignore Augmented Reality

3D-printed models have proven useful for education, planning, and communication among physicians and patients.  In recent years, more than 450 hospitals have started 3D printing at the point of care and have established a process where engineering teams assist with planning complex cases in a wide range of clinical indications.

A big challenge is the need for dedicated expertise, staffing, and equipment, which can add substantial treatment costs. This limits the number of patients that can benefit from its services. AR can support the use of 3D services for more patients. Segmentation, planning, and design remain the core components of any 3D modeling process, independent of how the 3D models are used or visualized. For 3D labs, this means their 3D printing process can also easily serve AR workflows.

Digital technologies such as AR and other extended reality (XR) technologies such as VR are ideally suited to process 3D models. Compared to traditional 2D images or 3D volumetric rendering, 3D models are relatively compact and easier to visualize and manipulate. Clinical engineers can highlight anatomical features that are difficult to appreciate on unprocessed medical images and create 3D visualizations that help treating physicians plan surgeries or discuss cases with colleagues and patients.

These similarities make 3D printing and AR highly complementary technologies whereby 3D labs can decide on a case-by-case basis which visualization approach is most suited.

While 3D printing has been actively adopted in fields such as orthopaedics, cranio-maxillofacial surgery, and cardiovascular disease, it is still underutilized in other clinical areas like trauma surgery and oncology. This is rarely because the value of using 3D is lower in those indications, but mostly because for 3D printing the cost can be prohibitive, technology may not be available or lead times could be too long.

XR technologies like AR can provide near real-time visualization of 3D preoperative and planned postoperative situations, providing valuable guidance for physicians managing complex trauma with short timelines. This contrasts with 3D printing, which requires lengthy processes and alignment of multiple stakeholders, both challenging in emergencies.

Also, regarding cost, AR can introduce scalability to new use cases for 3D modeling and planning for typically underutilized fields such as complex oncology treatments. Many indications involving neoplasms are considered appropriate for 3D printing. However, the cost of a large, multi-color 3D-printed model (e.g., of an abdominal, liver, or kidney tumor) becomes challenging. Some centers may not have the necessary capabilities to print such models in-house and may have to rely on an external service. Extensive design work may be required to make such models printable, e.g., if thin vascular structures are an essential part of the 3D model. In such cases, AR could be a solution to create a relevant 3D model for planning to bring to the treating physician but at a lower cost and effort. While some would undoubtedly consider having this in a regular viewer on a tablet or computer sufficient, others will appreciate the 3D perception, interaction capabilities, and ability to perform a task while interacting with the model as an added value of AR.

Certain clinical scenarios pose specific and sometimes complex design requirements for a 3D-printed model. For example, very small anatomies such as pediatric hearts may benefit from being printed at scale to increase the interpretability of small anatomical structures. In some cases, multiple prints may be required to compare different anatomical states, e.g., preoperative and planned anatomy. For educational purposes, complex designs may be made to make models more useful, e.g., by assembling different anatomy parts using magnets. The static nature of a 3D-printed model means that these scenarios add extensive upfront planning to realize the model’s full potential.

Also, a 3D print is limited in the information it can contain outside of the anatomical 3D visualization. It is possible to enrich the anatomy with relevant patient data, like a label for traceability. However, providing more detailed annotations (e.g., of anatomical landmarks or features) or transferring digital measurements (e.g., relevant planning measurements) is more complicated. The 3D print should be complemented in those scenarios with a digital report.

AR technologies intrinsically have more flexibility about how much information they can retain and offer the user. Multiple variations of a 3D model can be provided without additional cost. The 3D data can be enriched with annotations or measurements. Modifications to the models can be made interactively by the user, including the ability to zoom. Dynamic content can be integrated. And probably, many more options will be discovered as engineering teams start investigating how best to communicate a plan to the physician. They are just scratching the surface of what is possible.

In the medical field, accessibility to 3D models and plans during patient treatment is crucial to fully realize its benefits. However, traditional display systems often present challenges, such as poor ergonomics and inadequate interfaces. These can limit a physician's control over the 3D data, especially in the operating theater. To address these issues, AR can provide significant advantages by allowing physicians to organize the 3D information to suit their workflow and the room layout. Additionally, AR enables physicians to select or manipulate the data as needed, using gesture interfaces that eliminate the need for an operator to handle the visuals using traditional input mechanisms. Overall, AR has the potential to be a valuable asset in the medical field by empowering physicians with greater control and flexibility in working with 3D models and plans.

The educational benefits of XR technology are widely recognized. It can easily be integrated into a training program, for example, around anatomical knowledge, preoperative planning, or surgical skills. At the same time, the future growth of the technology will be driven by adequate training of experts. This aligns with some of the EU recommendations for the future of technology, where it is crucial to address the lack of skilled professionals by investing in training programs. Hence, hospitals with an educational program should start working with the current state of XR technology to improve their educational curriculum, spread awareness, and help future users gain familiarity with the devices and their applications. This will grow competencies and provide a head start when the technology matures further, and increasingly specialized applications become available.

Hospitals with research programs should also reflect on the opportunities to integrate this technology into their research programs. Many of the potential benefits are hypothesized and are currently under investigation. In the coming years, it will be necessary for hospitals to ensure the large-scale validation of new AR products, allowing them to become a sustainable part of future clinical pathways. Especially around operational efficiencies or the impact on patient treatment and outcomes, there is plenty of room to prove the value of XR technology for personalization.

Finally, working with the current state of AR wearable technology can also provide an opportunity for individuals and organizations to contribute to the development of the technology. Feedback from early adopters can help manufacturers and developers identify areas for improvement and refine their products to better meet users’ needs.

Extended reality — particularly AR — should be expected to play a significant role in the further growth of personalized care. AR will enable intuitive, easy-to-use, and immersive access to information to consult or use during the execution of a patient’s treatment. We envisage that AR technology will claim a big role, e.g., as a “personal assistant” to physicians for managing a personalized treatment pathway. The ability to add virtual information at the right time and place intuitively and ergonomically is extremely powerful.

The medical world has become increasingly complex and specialized, with more and more personalized data being acquired and processed. The demand for using this data in high-quality and sustainable care pathways continues to rise. We should further develop AR technology together so that it can claim its rightful place as a technology component that will help address complexity in the future.