Are metaverse applications becoming a reality in medical training?

M

edical training – and particularly surgical training – has long been a high-stakes game. Simply put, it’s difficult to gain familiarity with the body without access to actual bodies. That isn’t always straightforward when there are patient lives on the line.

As well as poring over medical diagrams, medical students are often required to dissect cadavers. This is expensive for the medical institution and grisly for the student. Further down the line, trainee surgeons must learn by doing, with all the attendant risk of medical error.

In recent years, however, another possibility has emerged. Taking their cues from the aviation industry, medical institutions have been moving towards simulation-based training, based on virtual and augmented reality (VR and AR). Moving forward, we are likely to see the introduction of an entirely new educational environment, located within the so-called ‘metaverse’.

“With no standard definition yet, the metaverse means different things to different people depending on the nature of their business,” says Rupantar Guha, an analyst at GlobalData. “However, GlobalData defines the metaverse as a virtual world where users share experiences and interact in real time within simulated scenarios.”

Within medical training, this means mapping the body in 3D, in precise anatomical detail – a world apart from the 2D images students have classically needed to study. It also means giving surgeons a space to hone their skills before they use them on a real patient.

“We have noticed that medical residents who want to specialise in interventional cardiology don’t have many opportunities to get hands-on experience,” says Dr Ioannis Skalidis, of the department of cardiology at Lausanne University Hospital in Switzerland. “In the metaverse, a lot of people at the same time can participate in operations and familiarise themselves with being the first operator.”

He adds that today’s medical students often begin their training feeling comfortable with VR applications.

“In contrast to the older generation, they're more familiar with video games and how they can control an avatar,” he says. “From the feedback that we have, we think medical students have a lot of interest in exploring and investigating this domain of cardiology.”

If this all sounds a little speculative for now, that’s unsurprising. To date, VR and AR are still largely the province of gaming. They have yet to impact most people’s working, socialising and entertainment preferences, let alone their interactions with their doctor. And it may be some time before medical students are routinely trained in an immersive virtual environment.

However, the metaverse is generating a significant buzz in tech circles and beyond – just witness Mark Zuckerberg’s decision to rebrand Facebook as ‘Meta’. According to GlobalData, the metaverse is set to disrupt several industries over the next three years, including retail, financial services and manufacturing. It could also have a huge impact on healthcare, with use cases across telehealth, therapy and virtual training.

“Despite being traditionally late adopters of new technologies, several healthcare companies, especially hospitals, are exploring opportunities in the metaverse,” says Guha.

He adds that the Covid-19 pandemic has spurred a revolution in telehealth. In July 2020, 95% of US healthcare facilities reported offering remote treatment to patients, compared to just 43% pre-pandemic. This has created a healthcare environment that’s more receptive than usual to innovations.

“Healthcare companies are increasingly adopting technologies like AR and VR to enter the metaverse,” says Guha. “AR- and VR-based services such as therapy and surgical training are gaining prominence in the market.”

While it’s early days for the metaverse, investment is on the rise. GlobalData expects the AR and VR market, worth $11bn in 2020, will surge to $204bn by 2030. Meanwhile, the metaverse itself could be worth $13tn by 2030, according to Citi Global Insights.

Within healthcare specifically, an early trickle of applications could pave the way for an influx to come. Already, we are seeing a number of striking examples, not least GameChange, a psychosis treatment involving a VR cognitive therapist, and VR headsets for people with paralysis or dementia.

DeHealth, a British startup, has created a Decentralized Metaverse based on AR and VR. This enables doctors and patients to interact in 3D and earn virtual assets by selling their medical data. On a similarly ambitious note, Dubai-based Thumbay Group plans to launch the world’s first metaverse hospital, which includes a metaverse medical university and a virtual wellness domain.

“The metaverse hospital will be completely virtual, where people can come with an avatar and consult doctors,” says Guha. “Thumbay also claims that the metaverse hospital could help boost medical tourism.”

Another major project is Aimedis Avalon, which bills itself as the biggest hospital and healthcare space in the metaverse. Doctors will be able to hold consultations here, examine patients and monitor them remotely. There will also be opportunities for professional training and new ways of doing research.

“The hospital has different parts, including various consultation rooms for medical visits, a virtual laboratory for virtual heart attack training, and a virtual cardiac MRI room and cardiac surgery theatre,” says Skalidis. “It is already completely ready for use and we’ve just used it to perform the first cardiology act inside the metaverse.”

As reported in their new paper, Skalidis and colleagues created a digital consultation room based on AR and VR, and gave the patient and doctor an Oculus Quest-2 VR headset. The patient also had a smartphone-based electrocardiograph device, which he used to record his clinical data. Using avatars, he presented this data to the consultant, who recommended he seek further medical attention.

“Up till now there have been many metaverse projects that are mostly theoretical. We’ve proven that it can be done,” says Skalidis. “While we can’t yet obtain all the information that we’d get from a face-to-face encounter, we can have ECT images, blood pressure monitors, heart frequency monitors and oxygen saturation monitors, all directly integrated inside the metaverse.”

On the medical training side, we are seeing a number of tech startups partner with large healthcare companies, in order to bring AR- and VR-based training onto the market.

“In 2020, Johnson & Johnson partnered with Osso VR and distributed nearly 200 Oculus Quest headsets to surgeons across the US,” says Guha by way of example. “Spineology, a spinal surgery device maker, has partnered with Ghost Productions, a VR surgical simulation developer, to integrate VR-based training for its sales teams. Spineology aims to educate sales personnel on its devices so they can improve engagement with medical providers.”

In terms of AR, companies like Aris MD and Echopixel are looking to gain a foothold in this sector. Aris MD offers 3D visualisations of patient anatomy, allowing the surgeon to perform the procedure in virtual space. Meanwhile, Echopixel creates a ‘digital twin’ of a patient using standard medical images, allowing doctors to experience that image as a ‘4D interactive hologram’.

While these applications are certainly promising, we should caution against getting ahead of ourselves.

“Much of the progress of metaverse-oriented medical training depends on the maturity of technologies like AR and VR,” says Guha. “Until then, the healthcare metaverse will remain niche and most-likely experimental. Many metaverse-enabling technologies are also still developing and will require time to mature and prove their potential.”

He warns that there are other limiting factors too, which could potentially stymie the metaverse ambitions of healthcare companies. On top of the massive investment required, some applications are likely to raise privacy concerns, while regulating this emerging sector could prove to be an uphill battle.

Skalidis believes these challenges are surmountable – costs are likely to come down as competition increases, while privacy could be assured through use of the blockchain. He concedes there are still big questions hanging in the balance, not least around the model of ownership (centralised or otherwise?), but remarks that the medical field has taken some critical first steps.

“Although it will take a while to apply the metaverse to the operating room, on the medical education side we’re already working on it,” he says. “We’re in the process of organising the first medical education lecture inside the metaverse for cardiology students – this will happen in the next few months.”

Over the following years, he believes a lot of different medical specialties, not just cardiology, will follow suit. There is certainly a long road ahead, but the metaverse for medical training is already more than hype.