Medical 3D Players: AR Is Changing the OR for the Better

Pieter Slagmolen  00:00  

Welcome to 3D Players. The podcast where we explore personalization in healthcare to advancements in 3D technology. We'll talk to leaders championing more predictable and sustainable patient care. I'm Pieter Slagmolen, my co- host is Sebastien De Boodt.    

Sebastian De Boodt 00:14  

Thank you, Pieter.    

Pieter Slagmolen  00:16  

Our guest today is Sebastien Henry, CEO of Pixie Medical, a French company active in augmented reality in orthopedics. Sebastien, in France you're considered a technology pioneer in personalized orthopedic care not only for the work that you're doing today at Pixie but also for the work that you've done early in your career, especially in personalized knee guides. It's great that you'll share more with us. Welcome.    

Sebastien Henry   00:43  

Thanks, Pieter for the nice introduction. My name is Sebastien Henry. I'm 50 years old, but I started 25 years ago in computer-assisted orthopedic surgery. Computer orthopedic surgery is dedicated to helping surgeons during orthopedic surgery to better position implants when your old cartilage is broken, and you need to replace that with artificial joints. We created 25 years ago a kind of GPS for orthopedic surgeons to help them better position implants. It was my first experience with a computer placed in the operating room and a surgeon trying to do their best to find a better alignment or a better implant positioning. Ten years ago, I created a new company still dedicated to the orthopedic industry—but using new technology: custom jig custom instruments created around patient anatomy. (It helped) to better position implants, by using an implant instrument dedicated to the patient. And four years ago, I created a new company. I think this is a new area for orthopedic-assisted surgeries; it is an augmented reality tool to better position the implant by using augmented reality glasses, but also a new range of planning and personalized planning, or personalized implants, in order to better help the patient.     

Sebastian De Boodt 02:25  

And where did that personal motivation come from? Because to really go into and improve procedures, you know, what triggered you to really build your career?    

Sebastien Henry   02:38  

I am not an engineer and orthopedics was an opportunity; it's not a choice. And I discovered a technology that can look at first glance as very traditional like implants made of metal or from the plastic industry. And I like innovation, even in my personal life. I discovered in this traditional industry, a new way of thinking and the possibility to develop new tools to better help patients have better outcomes after surgery. And I think, I try to be driven by the idea that technology can help patients.

Pieter Slagmolen  03:21  

So, in those 25 years, you also decided to become an entrepreneur. Basically, why did you decide in the end to launch your own company and maybe leave the safe space of a larger medical device company?    

Sebastien Henry   03:36  

To create a company that was an opportunity, because I was not hired for some ideas, but for developing those ideas. When it’s your idea, and if you cannot find people willing to do that, you are being courageous and going forward, and on your own to develop your idea. To be a pioneer is to do some things that no one wants to try, or wants to develop, so it's really exciting. And innovation in orthopedics is a nice place to develop your ideas.  

Sebastian De Boodt  04:22  

And again, to continue the history, at some point OneFIT Medical became a part of your imaging. Yeah, how did they get to that decision? And how was that change for you?    

Sebastien Henry   04:35  

Yeah, I created pick OneFIT Medical in 2011. And we had grown very quickly. One of my customers asked me to develop some planning tools. EOS Imaging is a French company created by a Nobel Prize scientist. They created a very sensitive CT X-ray detectors and developed the weight-bearing low radiation x-ray system. I had all the knowledge of how to develop software around this imaging system. And the combination of both technology imaging, plus dedicated instrumentation, and customized instrumentation, was nice because most of the imaging technology used to create custom jigs as CT scans or MRIs with less radiation in a weight-bearing position imaging system.  

Creating a custom jig was very interesting. And as a very small company, at the beginning working with an established company was a good combination for the company. So, working for EOS imaging, I was in charge of advanced orthopedic solutions. That was mainly creating a planning tool linked to this imaging system. We developed knee and hip planning; hip planning was interesting because we were able to work with a patient in a weight-bearing position, but also in a sitting position with the capacity to analyze from a patient-specific perspective where to have the patient move. Most of the time imaging is static. I think the possibility to position your patients was very interesting, especially for the hip when you can add a pelvic tilt. You can have a technical analysis of the movement of your pelvic region, from the spine and through your leg, in order to better create patient-specific planning. And after that, we created patient-specific spine planning and we use that for predictive models for a better alignment of your spine after surgery. So, personalization planning is to (basically) create personalized planning for your patient. And with EOS we had a new opportunity to create such planning for better patient outcomes.   

Sebastian De Boodt  07:24  

This is exactly why we were so eager to get you on this podcast because of your vast experience in all sorts of different technologies and trying things out, you know, to come up with a better execution—and also a better plan. It’s great to hear about all those elements where you were involved. Today you are focused mostly on augmented reality. And with your new company, Pixie Medical, why did you actually choose augmented reality as the technology to now spend all your time and energy in?    

Sebastien Henry   07:59  

Yeah, Augmented reality is a new world. And in this world, you can have a menu of options that will change the way surgeons enter the O.R.  This is a computer; a smartphone you can wear on your head, and with the potential to compute, to display, and perceive the 3D environment around you. Compared to traditional instruments that we used in the past, like traditional navigation systems use a tracking system to get only 3D information. Augmented reality glasses are a more immersive solution with all what you need on your head. But the main advantage of this solution will just arrive over the next few months. This is connectivity, because you can connect to the packs, you can connect to data to server, to an algorithm, to accessories. And I think, really, it will be a major change for not just orthopedic surgery, but for all medical areas. And probably more. My focus is on the medical areas, and especially, the orthopedic. 

That's why Apple is launching next year, new augmented reality glasses and they say it will replace the iPhone. So, it's not just a tool. It's a new way of thinking with the potential. Now 20 years ago, we had, I remember the cell phone that was just to create a phone call. Right now, you have a smartphone with all the potential to detect the 3D environment with the new sensor and to create as many features that you want. You would like to use it like a Swiss knife.  

I really think that it's a new AR era for orthopedic navigation. And we started with a tool to add surgeons to navigate their total knee replacements. And we created our own algorithm to perceive our environment just by using very small cameras... there's a new one on the new smartphone. So, the first idea was to create a smaller instrument, that surgeon will fit with zero footprint. And I would say that it's a success because we sold in less than one year more than 100 systems in Europe and in Australia. And we did the first surgery in the USA. So, it's interesting.    

Pieter Slagmolen  10:40  

It's clear to me that the impact of this technology is pretty big nowadays. And the Apple reference that you make, I find interesting because I also believe that as more of the big technology players come in, that technology will be pushed forward and even create more opportunities. I think the value of Apple just on a single day went up by 100 billion, because there was some announcement and buzz on what they were going to be doing in augmented reality which is pretty exciting. I hope that we see similar things happening in our industry of companies moving forward, like Apple is intending to do with this. Exciting!   

I was curious (because you mentioned that in April) and you plugged something that you have in your pipeline, more related to the fact that it will be connected? Can you explain what that will mean? Why will this be so revolutionary? What will be the exact difference?   

Sebastien Henry   11:40  

Right now, we are using only the power of the computers that is embedded in classes to compute the data that we can get from sensors. From the camera, from the deep sensor, from accelerometers, and we can display that in a screen just to our screen, and we can create overlay on the patient. This is the traditional way to think about augmented reality. 

Connectivity is a matter of data. And multiple ways to think about data that you can access, you can create your own data by using navigation. And that data can help the next generation of patients by using that preoperative data to improve the operative outcome—if you are doing this the right way. You can analyze patient data before the surgery, after the surgery, but during the operation you need to collect data. And if you can combine that all together, you can create a new a new model that can help surgeons create the right decision. So, it’s being part of the connectivity of this new world of data that can be created new patient model. And why not a predictive biomechanical model? Or the predictive outcome model is something very interesting. It's a first idea of what you can do with connectivity. The capacity of glasses are limited, it's still a smartphone, very powerful. But if you want to create very powerful algorithm, you could need to use an external computer, or like an external server to use big number data or a powerful algorithm with a strong computer. And with connectivity, especially for that. 

Now that they're already connected with wifi or Bluetooth, what we need is a very high speed connectivity if you need to exchange a high volume of data. And 5G! When I think about connectivity, it’s especially with 5G. Nevertheless, it's not so easy because if you are working in the medical environment, you have a cybersecurity risk. If you open those channels of communication, we have a risk that pirates could enter into our system, and it could add risks for your patient. You have a risk of losing patient data. And it's something that we are very sensitive about. Everybody is sensitive. So, besides the technological aspect, you have to be open to risk and that's something you need to be careful with.    

Sebastian De Boodt  14:30  

So, if I understand correctly, you don't just see AR as a way to translate planning into AR but also to capture data during surgery. That could then let's say improve planning for next patients and you know...   

Sebastien Henry   14:45  

Well, exactly that is just one way to think about data that we have another way just inside the operating room. You have a lot of other computer O.R. systems and, I will say probably not the best ones. The one in orthopedic that everybody's talking about is robotic. Robotic is nice. But it's not the same to think about robotics in orthopedic industry and in the solar industry, like the DaVinci, the robotic trunk robotic system, a robotic arm with the navigation system. But the navigation STEM is the old-fashioned navigation tool, because it was created 20 years ago.  

One way is to think about those accessories, like the robotic arm, is to be connected with your glasses. With your glasses–when I talk about glasses, it's not just the glass that’s the computing tool–it’s the connected tool and the display with the state-of-the-art system, you can control a robotic arm, you can talk, you can exchange data with your instrument, if it’s equipped with an RFID chip. So, it's a new world where information can act wireless, you can control other systems, and you can talk to the world or aim to create a digitally augmented surgeon... not to give the surgeon a lot of electronic tools like a robot, a computer, and so on, it’s just to give them additional power like a Super Arrow. Super surgeons that will use new sensors. We pre-compute very complicated algorithms, like Amazon, to make the right decision for the patient. Planning is part of this project.    

Pieter Slagmolen  16:40  

I think it's interesting the way that you talk about this, it feels like you're saying for us that AR is going to be the central technology that kind of coordinates everything around it. How do you see that? Is that how you see it in five to ten years, if you look at all the technology that are around 3D printed instrumentation? Maybe patient-specific implants, planning, robotics, do you see AR as the central technology component that binds all of these together?    

Sebastien Henry   17:09  

Yeah, not just for medical areas. And I can tell you, I was an early adopter of the Google Glass and it was a failure. So we need to learn from this failure 10 years ago. But we evaluate and the technology now is more powerful, you can have very light glasses. And I really think that it will be the major change: connected glasses in the future. And it will be the central point of everything in the O.R., you can still have a robotic arm in your operating room, but it's very difficult to move it from one operating room to the other one, it’s a big capital investment. If you can add the computer on your head, if you can move with this computer, if it can help you for every step of your surgery (or before and after to analyze the patient and patient motion). If you want to follow your patient, it will be, I don't want to reach and plug a chip in surgeon brain! But it looks similar, an easy way to access data to make surgeons more powerful. They are experts, but they can find a lot of opportunities already. If you compare an orthopedic surgeon using augmented reality, if you are student using a robot, the fields, they look like an assistant of a robot. So, it's interesting to see also the personnel point of view will be our surgeon. I will feel good to have all the information and to adjust to a robotic arm as my assistant.    

Sebastian De Boodt  18:43  

I feel a lot of enthusiasm and excitement around AR in this room—and Materialise. Pieter is also the one who is championing AR, so I think I’ll try to be a little bit the critical mind in the discussion with a provocative question. I think the first time we met, Sebastien, was about 10 years ago. And then you know, we would probably be having the same conversation but then on knee guides and like we, you know, you were in that industry, and together with Materialise, championing this technology. And I remember those also as very exciting days to be to be in that part of that game. You know, all the surgeons wanted it all the medical device companies were looking at how can we get this as quickly as possible into the market? And then yet if you look at it today, you know, it never really reached a penetration in the market of more than 10% of the procedures happening with surgical guides. Do you know why that is? And why will this be different for augmented reality?    

Sebastien Henry   19:43  

Yeah, I, as I said I had in my 25 years of experience even standard navigations that failed to be a standard of care. The reason why? It was complicated to use. And you still add additional cost per surgery. That was a little bit expensive; capital goods are disposable. And in the last decade, you have two new families. So, on one side you have a robotic system that is an evolution of the classical navigation with a robotic arm. But I think we did this the wrong way. I'm not against robotic,  

20:32  

So why in your opinion is it that personalized knee guides never reached a point where there was a market penetration more than 10%? And why will it be different for AR?   

 

20:59  

To make the answer simple: patient-specific guides were interesting compared to traditional navigation because it was easy to use. And it makes the surgery simple. Compared to the previous way to do traditional surgery with standard segmentation or with navigation, it failed for two main reasons. The first one, the logistics were complex. You need preoperative images; you need to do a planning; you need to have your patient data out of the hospital; you need to send your data for external treatment. But the main reason is that you, you are limited in your function. It's if you want to realize your planning is perfect. But something can happen during surgeries. And especially, as I said before, a patient is not just a patient, its a patient with images. With a patient lying in supine position in the CT scan, you need to analyze motion. Or especially for the knee soft tissue balancing is part of your knee surgeries. And the failure of patient-specific guide, especially for the knee surgery, was not being able to adjust your planning in the O.R. according to specific information. You can recall or see during the surgery, I think this is a main, fairer, nice solution, easy to use, but with too much limited function and logistics that were a little bit complex.    

Sebastian De Boodt  23:03  

I think the advantage that you pointed out with AR and making the surgeon the superhero, because it gives him additional information to make important decisions during surgery. And that's maybe an important difference with knee guides, right where you have a great pre-op surgery, but then during surgery, you don't have that many options to deviate from your plan.     

Sebastien Henry   23:35  

This is what I think. It's more than just to be able to deviate from your plan, but part of the flexibilities that can offer augmented reality compared to the patient specific jig. It's something that is really an advantage, not for just for augmented reality for robotic, also you can change your planning in the O.R., but you need to collect that information during the surgeries. And also, when you have a planning, it's complex most of the time; it's a complex algorithm. If you want to change one position of your implant, it can have an impact on many parameters of your surgery. To be able to compute that in real time with the new information you can get during surgery is also important. Planning I think is still important, but I think in the future, we won’t talk about preoperative planning, planning will be integrated in the O.R. My dream is to proactively have images to get the images directly in the O.R. and to automatically reconstruct the 3D model from your images—without external manipulation. 

To have planning directly on your patients, the next generation of projects will not be seen on the 3D screen, or your virtual implant on a 3D model of your scan, but directly on your patient to see your 3D virtual implant on your real patient! This is the future! So that makes surgery easier. So no, you won't lose time before the surgery, you can if you still want to, but you could have surgeons willing to take time to prepare their surgery. But I think also that in the future, automatic planning will add surgeons to do the planning, and they will just fight you with their expertise to adjust the planning to the patients.   

Pieter Slagmolen  25:43  

As Sebastian said, I'm a big believer in this technology, as well, as I think for the potential benefits in logistics, in terms of the ability to adjust and to do some things intra-op. I think one thing that that is maybe a bit contradictory, and I'd like to get your view on, is that you said one of the challenges for guides was the need to have data go out of the hospital? How do you see that? Maybe the dynamics in the market have changed because in your vision of where AR will go, I see that interaction with data going out of the hospital going out of the O.R. as essential to realize. So do you think that the minds of the market have changed? Or is there something else playing here?    

Sebastien Henry   26:27  

I think hospital wants to protect patient data SQL system, or to protect SQL data. To put data outside of the hospital will be complicated. Patient data will be complicated, but to analyze data in order to create predictive biomechanical models outside, that will ultimately help a surgeon to make decisions during the surgery is still my vision. I don't think that you need to export patient data for your current surgery. What I would like to do is to explore your packs with your virtual desktop to go back in selecting your preoperative data. Why not just preoperative data to automatically reconstruct? Materialise is doing that well. To construct automatically with a computer that is on your glasses, and to get in real time your 3D model, to match the 3D model to the real patient, to automatically get your planning, and to realize your planning with additional data you can get on your patient. That means soft tissue balancing, or any kind of data you can get from your patient’s motion.    

Pieter Slagmolen  27:46  

I think it's an interesting view indeed to disconnect maybe personal patient data and personal data from data that you might need calculations. They're not necessarily one and the same and indeed there might be a different expectation or requirement in the market for those. Yeah, I think I totally agree with that, actually. Nice.   

Yeah, maybe, a good time actually to ask where you're going next because we talked about personalized guides. We talked about Pixie and the focus on knee surgery, orthopedic surgery specifically. What's next after the knee for you, and why do you want to go in whatever direction that you choose?    

Sebastien Henry   28:31  

As I said, when we discuss about patient-specific guides, we were thinking that it was the perfect solution. And we discover, time after time, that it was not the perfect solution. So probably augmented reality will evolve; also, the technology is moving so quickly that we need to be open and to be ready to see what will arrive in in the next three–four years. I think it's the first time in my personal life, professional life, I can just see opportunities... it’s this new technology, with standard navigation, with robotics, with cutting guides. Very quickly, we were able to see the limit, not that the limit of technology was nice, but evolution by using this technology was limited because of the connectivity, because of the display, because of the computing, because of everything that is the heart of the of this technology. I have a feeling that the potential is open and nobody is about to see the limit. This is what is interesting. Now, I think inside, what is visible? I have enough to do for the next 10 years. So, it's good for me. I'm 50, so I will do probably all my, the rest of, my professional life in this area, but still stay open. Don't look too close, like, most of the people making a big mistake by seeing augmented reality, just as a display. No, it's not just a display, it's more. And if you can just open your mind to the potential of this technology out of the display, you are already at the next step.    

Sebastian De Boodt  30:18  

Yes,  the best is still to come for AR. Think it also has to do if you compare robots developing a robotic system, I mean, it takes so much time and money! So, it's really something that only a few can pull off and be successful with, which is also what we see happening in the orthopedic industry at the moment, where you have really the big ones coming out with a robot, while with AR, it's way more accessible. And it's, you know, you see so much startup activities around AR. And I think that it will be much more competitive space in terms of pushing the boundaries of technology.     

Sebastien Henry   30:58  

Very important is also the cost of healthcare system; it cannot absorb all new technology at any price. I was thinking that, so we just decided to work with off-the-shelf glasses—and I have a computer. Competitors decide to create their own glasses really specific for this technology. But you should really limit this model, because you need to have healthcare systems that are not able to pay for that. You can still have the US market try, and some European markets that can access high-cost technology. But I was thinking that right now by using off-the-shelf technology, like the smartphone, and help to develop AR glasses, it’s the same kind of technology. So, it enables you to reduce costs. For one robot, you can buy 100 augmented reality system and the clinical benefit looks like the same right now. Potentially. And if you want to create a new standard of care, you need to be sure that everybody will be able to use it. We have a lot of progress from India to China. And this is a country with healthcare that needs to work with a lot of people and without the same model of reimbursement that we have. That's why also we decided to create a new business model. If you are close to the software, you can create a business model without disposables. All the systems we had in the past were capital goods–expensive plus disposable– so you need to pay per surgery. Even the PSI which is a patient-specific instrumentation system, the custom jig, was a single-use project. And we are the first solution without disposables. We created instruments that can go to steam sterilization. So, it's also new, and you pay for the service when you use the product. The evolution is not just in the technology, it's also the way it will help you to create a new standard of care by reducing costs, still with the same patient benefit for the patient.    

Sebastian De Boodt  33:20  

And in trying to create a new standard of care, as we all know that robotics are, yeah, seen as also a big uptake and pushed by big industry. Do you foresee an epic battle in the coming 5–10 years between AR and robotics trying to outbid each other, or will it be more in synergy as you said before, where augmented reality might be just a way to better inform robotic surgery or things like that?    

Sebastien Henry   33:48  

Robotics is a big world. And there is confusion most of the time because most people are seeing robotic as a component of tracking, of display, and planning. That is very important also in robotics and robotic arms. But what we try to define with this new AR system is the component outside of the robotic arm. My vision is, and I'm sure that it will happen, is our system controls a robotic arm. Any robotic arm will be an option. What is interesting about robots is being precise in your surgery. That you can do that with the maturity system and if you need more precision, and I'm not sure that for now you need this level of precision for the orthopedic, but anyway, if it's important for the surgeons, you can combine augmented reality navigation system with a robotic arm. It's not an opposition, but it's nice. It's really nice to see the robotic system worldwide. I am just concerned by the additional cost. We have our surgeons make a calculation, that by using a robot, the additional cost per surgery is €3500 per surgery, (or) more. And for now, I have to say that I don't see a clear benefit of this additional cost. You're gonna have benefits, but I cannot see (them).    

Sebastian De Boodt  35:25  

That is an invitation for someone from Stryker and Zimmer Biomed to join us on this podcast, and next episode, because I think it's a very fair argument. I wonder what their answer is to that.     

Sebastien Henry   35:41  

That's the potential to prove that it's efficient, and probably also working for the next generation. But for me, it's a tool to develop, it's a platform to develop, probably the next generation. Currently, I don't see that. And probably augmented reality, also, at the first level, is a tool to introduce this technology. I'm not against robotics, and I think that the future probably will be for specific joint surgery to combine both, but you cannot find the advantage it boasts.  

Pieter Slagmolen  36:17  

I totally agree. I think it will help sustainability in the end if you can eliminate disposables. If you can make if you can build a more generic hardware with glasses and then use the robotic arm where it might be most useful. In those cases, you need very high accuracy. I think in the end, it will help the scalability. It will help reach more patients with preoperative planning, also, with proper execution, I think it's very important.   

Sebastien Henry   36:42  

What is important is the patient benefit. In the end, that's it. It's not just to have a tool for fun; it will have an impact on the patient. And this is where we need to be; it's not just a toy, it's a tool to help surgeons to perform better surgery. And I think one big advantage of this new technology is to be less invasive. At the end to be more precise in your surgery. It’s one point is to be less invasive, another big advantage of this technology.   

Pieter Slagmolen  37:21  

You are a mind-reader, Sebastien. We actually had on our questionnaire to ask where the border is between toy versus tool, and you actually use the same analogy yourself. You're reading our minds on what we're thinking. Awesome. Okay, great. I think it's time to close off this episode, which was a very exciting discussion. Maybe I'll leave the final question to you, Sebastian. So, the provocative question, maybe, or a very futuristic question.   

Sebastian De Boodt  37:57  

Well, I'm just curious, you’re, let's say, in the midst of your career, and let's assume that there's a very unfortunate situation that arises where you will need at some point, maybe 20 years from now, a knee implant, what do you hope that industry can do by that time that’s not possible today?

Sebastien Henry     38:22  

Still have more expertise on knee. And one issue with knee is that 30 to 40% of people are thinking about their knee every day—that's not the case for hip, for example. You can forget your hip, for the knee, you have 30% of these that are not forgotten. Everybody’s dream is just to have a “forgotten” knee. It's a combination, not just an implant design. It's a combination of multiple factors; the operatory surgeries that can help you psychologically think less about the outcome and about the surgery is one aspect. The less invasive surgery is another point. Patient-specific instrument implants is a new way of thinking of an implant for patient, but for now, we don't have proof that it has clear clinical benefits for the patient. But my dream will be to have a knee laboratory with the best fitting implants with a predictive biomechanical analysis of this knee according to my specific anatomy, and to do that with a minimally invasive surgery. This is what I would expect.   

Sebastian De Boodt  39:47  

That's a nice expectation to have. And shows there's still some work to do for entrepreneurs and innovators like yourself. So, thanks a lot, Sebastien, for this really energizing conversation, for talking to us about your vision on augmented reality. And in healthcare! We learned a lot of things and that AR will be there to make surgeons be like superheroes, but that it will also be a technology that really allows us to drive down the costs of healthcare. And you know, with that hope, we're two, three, maybe ten steps closer to making sure every patient can forget about their knee 20 years from now. That will be fantastic. We highly appreciate your sharing your insights with us on 3D Players, a podcast where we explore trends, insights, and innovations in personalized and sustainable health care. We are your hosts Sebastian De Boodt and Pieter Slagmolen. We thank you a lot for listening and hope you’ll join us for the next edition.