Shafi Ahmed is a cancer surgeon at the Royal London and St Bartholomew’s Hospitals and, thanks to the global virtual reality live streaming of one of his operations, is also the most watched surgeon in human history.
The award-winning doctor, teacher, innovator, entrepreneur and evangelist for augmented and virtual reality was the closing keynote speaker at Digifest 2018; here he shares his thoughts on the power of virtual reality, robots in the operating room and the future of medicine.
More than five billion people around the world do not have access to safe surgery. How do you see virtual reality and other technologies changing that?
The first thing that these technologies allow us to do is to connect humankind. The 3G, 4G and wifi connections we have around the world are quite simply connecting people. From there, virtual reality (VR), augmented reality (AR) and other technologies allow people not only to connect but to share knowledge on a broader scale.
Conventionally, surgeons are often teaching or training just one or two people who are in close proximity to the surgeon in the operating theatre. I believe that knowledge could be shared much more widely with many more people. Wouldn’t it be great to share it with tens of thousands rather than a few?
That’s what I’ve been doing with these technologies – VR and AR – that allow people to be immersed with me in my operating theatre and allow knowledge to be shared around the world, scaling up education and training to a global level. It provides one of the solutions for the deficiencies and inequalities in health care around the globe.
What about closer to home, in the UK: what’s the benefit for surgeons and students here?
Looking at where we are in terms of education, we see a continuum.
Let’s go back to the beginning – thousands of years ago we were drawing on stone walls, some of which still remain. Then came papyrus with the ancient Egyptians and then the printing press - Gutenberg was a seminal moment in the dissemination of knowledge. Now we have e-books and e-learning on online platforms.
AR and VR are an extension of those new ways of learning. Our students and trainees are digital natives, they can access information much more readily and they need to be taught in different ways. Learning itself is going to evolve in different ways and I think the UK will benefit hugely from access to AR and learning platforms.
There are a number of areas in which we can use virtual reality in learning here, particularly in medicine. One of the basic areas of learning is the examination of patients, and that could be done much better virtually, with added assessment and gamification. In my field of surgery, it’s about replicating operations and simulating them in VR and AR.
I think the whole world of simulation will change when we add haptic feedback in the future. There will be interaction and assessment tools within the VR environment allowing people to simulate not only surgery but all procedures, such as drug taking, putting cannulas into patients, taking bloods, putting in gastric tubes or whatever.
What opposition, if any, did you face when you first brought VR into the operating theatre or suggested bringing it in?
I’ve been very fortunate in that the hospital I work in, The Royal London Hospital, which is a part of Barts Health, the biggest hospital in the UK, has been very supportive of innovation. If I have an idea, they haven’t said to me ‘go and think about it for a while and do the due diligence round it’ because when that happens it delays things or simply stops innovation in its tracks, because innovation has to move quite rapidly. Instead they’ve said, ‘great idea, let’s look at how it works, let’s take the hospital and patients on this journey, let’s talk about the ethics and the confidentiality issues that might ensue’.
We talked to the legal team to ensure that we were doing the best that we could and mitigating the risk as far as possible. So the hospital has been very supportive of innovation and that’s been very helpful for me.Watch Shafi's closing keynote at Digifest 2018
However, other people are nervous of innovation. You have the inventors and the early adopters who are going to drive change and, in medicine, sadly those people are in a minority and most people are slow to adopt new technologies and adapt to change. I think that’s one of the problems we have in medicine as a whole. We accept dogma and tradition. I want to challenge that on a daily basis and part of my work is to show that we can challenge it in a way that’s effective.
Then there are the big global companies and we need to help them shape and support technological advances in a way that’s safe and considered, particularly when it comes to health and patient safety.
The other thing I think we need to think about carefully when bringing innovation into our setting is the end user, the patient, who is most important of all. I’ve always been very clear about taking patients on the journey with us, to make sure they have consented properly and that they understand what you’re trying to achieve. You’d be surprised at how kind and amazing our patients are when you are trying to do something to help humanity and to improve quality, safety and outcomes. Patients are our partners in the hospital environment and, throughout the whole process of innovation, you have to take all those people with you otherwise it will never work.
What has the reaction been from the patients involved, their families and patient advocacy groups more widely?
Moving forward, VR will be very useful for education purposes for patients. Imagine, for example, that you can be connected to VR as a patient to see how you will be treated – say, going from the ward to the operating theatre and back again. That’s the kind of thing we’ll be doing in the future.
Patients themselves are very warm to that – many of the patients that I have operated on while recording it in VR have later watched their own operation in VR to see what it showed. I used to think that was quite strange but then I thought why not? We often give patients videos of operations or pictures and this is just the next step. Patients have been very interested in this process.
Ultimately, what patients want is openness and transparency but for years and years we’ve been shying away and keeping the operating theatre private and surrounded in a kind of mystique. This is opening it up and making it more transparent. Ultimately, we’re all human beings and people can see that.
Live screening and live operating has its own issues and I think that, for training medical and healthcare professionals, it is entirely appropriate. I think the issues around live operating for viewing by the public can be more controversial. I am keen to demonstrate the power of what we’re trying to do and to showcase new technology and I think we’ve had very good feedback. I’m also cognisant of the fact that patient organisations might not think it’s the right thing to do.
Most organisations are risk averse and it’s challenging to think about how technology can be used in this way because it is new. I think it is empowering for people but I also think we still need to think very carefully about how technology might be brought into that kind of scenario.
Do you think that being able to see their own operation afterwards helps patients to feel a greater sense of empowerment around their own bodies in what can feel like a very disempowering situation?
Yes, certainly. Additionally, when you’re watching it in VR it’s not just about the actual operation, which you can watch on any YouTube video. It’s about the whole environment, the team working around you, seeing just how many people are involved in improving the outcome of your treatment, seeing how the whole system and team around you is working, not just the surgeon. I think that’s quite important.
When I did the live VR that I recorded two years ago, with the explicit consent of the patient and his family, I came out of the operating theatre and met the family – the wife and son of the patient. I told them that the operation had gone well and I was happy with it and they replied that they knew because they had watched the operation live, in VR. I was quite shocked. I hadn’t expected them to watch it live but they said thank you, it had helped them to get through the procedure.
Normally when your loved one has an operation you sit around worrying about what’s going on for hours on end and go for cups of coffee, so they said that they felt reassured that, while their loved one was having the operation, they were watching it, seeing what was going on, the ability of the surgical team and could see that he was doing well. I think we underestimate how people will respond and assume they will respond in a negative way but I’ve been surprised by the positivity of patients around these kinds of things.
You’ve used Snapchat Spectacles and Google Glass for this work. Does the type of technology you use affect what you do in any way? Do you behave or respond differently, beyond performing the operation in the usual way?
Now that we’re used to doing live recordings for training purposes, whether using Snapchat or HoloLens in the theatre, the team know how to behave – you are on show, you are visible, people are learning so you have to make sure that you are exemplary in the way that you work in theatre.
When we first started we didn’t really know what to expect but now, with the different platforms, we try to engage through each platform. So, when you’re doing Snapchat it’s quite different and I was nervous when I first started doing that, I wasn’t sure how that would pan out on social media but I was pleased that people do interact. Interaction is important and I have a moderator, who is my trainee in the operating theatre, who makes sure he tracks the discussion and the questions so it’s a way of interacting, not just showing a video.
Similarly, with Google Glass or other kinds of glasses, we have a system where we moderate and make sure that there’s a learning around it so it’s not just viewing that’s going on but interacting with people around the world – they ask questions, we share information. Remember, when we’re operating normally without the glasses and the technology, we’re teaching anyway – I’m a teaching surgeon so I’m used to constantly explaining, answering questions and training my juniors. All that is different now is that it is being transmitted and there are interactive elements within that.
So it’s taking what you are doing already and moving it on. I’m not sure it’s for every surgeon, it’s different, you’re engaging with the public and you have to be very careful about making sure that you’re professional in your work, that the whole team is working well and have an understanding that you are being broadcast and teaching people on a global level. It is also important to showcase what the NHS is doing and pioneering.
Beyond VR, what’s next? How do you see this work developing, in the short term and looking a few years further ahead?
First of all, VR needs to create enough content. At the moment is it largely hardware driven and content is at a premium so we need to make sure that content drives VR. Once you’ve done that, you can engage a community that improves over time. You also need to create a learning environment. It’s not just about the video, it’s about the whole learning environment – you navigate through a platform, you learn, you have assessment tools, you get learning materials as well as the video. That’s what we’re trying to create.
Going forward from that in the medium to long term, it’s going to be about adding photo-realistic imagery, avatars that look like real people rather than the cartoonish ones we have now. Then there is haptic feedback – creating a realistic sense of touch and feel, and that’s going to be a game changer for a lot of people.
We’re not quite there yet, a lot is promised, but for me it’s a no-brainer to try to create a device, like a glove, with a motion-sensing capacity in virtual reality and a sense of touch so you can do a virtual operation if you need to, or a procedure, using that tactile feedback that you get. That’s where we’re heading towards and, fairly soon, it will become available.
The other area is mixed reality. When you work with people from across the globe, you’re having to physically transport yourself. Wouldn’t it be a good idea if you could put on a HoloLens or other mixed reality headset and virtually transport yourself to an operating room in the middle of Africa somewhere? You’d appear as your avatar, looking like a real person, walk around the operating room, have a look over the shoulder of the surgeon, give advice and then leave again. It’s not quite hologram but, rather, holo-transportation and that’s where I think we should be getting to. It would also change how we connect at conferences – instead of having to travel to conferences, you could immerse yourself in that environment virtually.
The way we undergo clinical practice would change, the way we connect with our patients in the virtual world and the way we connect with our colleagues would change – we could discuss cases with different people from different parts of the world in a virtual space. It could also help with surgical planning – you can walk around the ward and access information being broadcast to your headset. In the operating theatre itself, you could access artificial intelligence that gives you clues about what you’re looking at, ideas about strategy in the operation, and quantifies all that with data that allows you to improve your standards. Those are all things I see happening in the next five to 10 years.
You’ve also talked about ‘surgical singularity’ – can you tell me more about that and what it means? And when will it happen?
When Ray Kurzweil wrote his book The Singularity is Near (2005), about the point at which robots will become as good, if not better, than humans in function and behaviour, he said that it would be about 2030 and then pushed it back to 2040. In the intervening years we will see the rapid advance of computational power.
Surgical singularity looks at the point at which a robot with an interface can perform an operation as well as a person. As a surgeon we take history from a patient, we use knowledge and experience gained over 20 or more years. Computers can learn more quickly, so at what point will they be better at making that diagnosis, picking up on the clues that are there? Then, would you trust a robot to perform an operation?
This is the year of robot wars – there are about ten of them coming out into general surgery to support and augment our practice. As they become smaller and more intelligent they will become better at supporting us. In the next 20, 30, 40 years could we have an autonomous surgeon working in theatre? It is very possible. It’s exciting.
I think it is up to clinicians to understand the limitations, and control what we do in the future, to work in partnership with the big global companies and startups to figure out that future for ourselves and make sure that we do the right thing ethically and morally and that we are improving standards. I think we should embrace it to get improved outcomes. If it democratises training and clinical practice around the world then that’s a good thing.
What has surprised you most as you’ve been doing this work?
The pace of technological progress. This has been called the era of the fourth industrial revolution and I think that really applies to medicine. In medicine so many developments are coming together – AI, robotics, blockchain, AR, VR, nanobiotechnology, sensors, big data. All of those things have come together quickly and in the last couple of years we have seen such immense change.
It’s our challenge as clinicians to see how we bring that all together to have a positive impact on healthcare. The sheer pace of change has been surprising but also incredibly exciting at the same time.