Feeling confused by the mounting hype around the metaverse and wondering what it might mean for your business? You’re not alone. When announcing the intent to “reach a billion people, host hundreds of billions of dollars of digital commerce, and support jobs for millions of creators and developers” Facebook founder Mark Zuckerberg explained in a blog posting that the defining quality of the metaverse will be “a feeling of presence—like you are right there with another person or in another place.”
Facebook, which has changed its name to Meta, is making a $10 billion bet, on the metaverse. Other businesses are still trying to figure out what this immersive universe might mean to their futures. Some are snapping up virtual real estate so as not to miss out on a new way to engage with customers; others are selling digital fashion items for avatars or making tentative attempts to get employees to collaborate in new ways.
Will any of this really be a game changer for industry? A look at Level Ex, a Chicago-based company that creates medical video games designed to advance the clinical skills of physicians and surgeons, gives a glimpse of more potentially game-changing applications.
The company, which works with most of the top 20 medical device and pharmaceutical companies, was purchased in 2020 by Munich-based Brainlab, a company that creates software-driven medical solutions that digitize, automate, and optimize clinical workflows for neurosurgery, spine, trauma, cranio maxillofacial, general and vascular surgery as well as radiotherapy and radiosurgery. Core products center around surgical navigation, radiotherapy, and digital operating room integration.
Level Ex and Brainlab are already leveraging advances in everything from spatial computing technologies to video conferencing and 5G’s greater latency to allow virtual surgeries to take place inside an operating room at the same time as a real one.
“You can project a virtual twin of the patient on a screen in the corner of the operating room and show the surgeon that it is better to do it from another angle,” says Sam Glassenberg, Level Ex’s Founder and CEO.
The technology is also being used for training. Two people thousands of miles apart can jointly practice virtual surgeries. (I participated in a demo over Zoom that allowed me to use a surgical tool to open the right side of a virtual patient’s knee to prepare for surgery while Glassenberg simultaneously pulled back the skin on the other side). Level Ex is also helping NASA build a framework for future astronaut medical training.
“We are reaching an inflection point on immersive technology, but the real value is being drowned out by the hype,” says Glassenberg. “We are doing something way more relevant with way greater economic impact than the types of applications you are hearing about for the metaverse. Healthcare is a multi-trillion dollar industry. The value of what we are doing is super high and you are seeing it right now, today. 2022 will be the year when surgeons in almost every specialty can be trained using any device over Zoom to simulate surgeries remotely and collaborate in the Cloud.”
The ability to use regular video conferencing to jump in and impact a surgery is new, says Glassenberg. “I will leave the decision on whether that is technically part of the metaverse or something else to buzzword bingo, but I can tell you that we are already using technology to create life-like digital twins of humans, realistic digital versions of real-world environments and enable collaborations at a distance.”
Professional Applications Are Leaping Ahead
That’s a big contrast to Silicon Valleys’ efforts to create the metaverse. A company called Second Life promised to change the way we work, play and purchase goods by creating an immersive universe in the early 2000s. Millions of people flocked to the platform, but interest eventually waned due in part to clunky interfaces. For its foray into the metaverse Facebook is betting big on VR headsets which are cumbersome, expensive, and potentially dangerous. (A recent story in the Wall Street Journal talked about how people using VR headsets are hurting themselves and landing up in hospital emergency rooms). Magic Leap, a company specializing in augmented reality and computer vision that is powering some of what Brainlab does, has raised over $2.6 billion in funding but has struggled to commercialize a headset at scale for either the consumer or enterprise sectors. And BusinessInsider reported earlier this month on disarray and uncertainty in the Microsoft division working on HoloLens, that company’s version of mixed reality glasses.
“As we’re seeing with HoloLens now, and we’ve seen all the way from Google Glass to Magic Leap, is that providing an entrance into the metaverse just isn’t that simple,” says a February 4 blog post on OnMSFT.com, a website built by a Microsoft-focused community that is not affiliated with the company, “HoloLens had field of view problems that literally made people sick and there was a privacy backlash from Google Glass that effectively doomed it as a consumer ploy, at least for the time being. Magic Leap (and possibly all metaverse hardware up until now) promised much more than it could deliver,” says the post. “This transformation to ‘“the next Internet,’“ a metaverse where we’ll all exist in an ‘“embodied presence,’“ isn’t going to be easy, if indeed it happens at all.”
But as Level Ex’s offerings illustrate, those constraints are not holding back important new professional applications. Brainlab’s Mixed Reality Viewer technology can already turn almost any work area into a virtual, interactive space for viewing a broad range of anatomy. Built in sensors in a head-mounted display scan and map the room, so that doctors can mimic a planned surgery, placing 3D anatomical models and 2D image slices where they want in their space, whether that’s an office, a meeting room or at home. The system allows for up to four people to collaborate and interact with each other in real time, either in the same room or remotely within the same network.
Over 200 systems have been sold around the world to hospitals leveraging Mixed Reality Viewer from Brainlab on Magic Leap’s spatial computing platform. The interactive 3D patient data viewing software is a functionality of the German company’s Elements Viewer and can be launched with just one-click, offering what the company says is a streamlined user experience. The mixed reality functionality delivers on several levels including surgical planning, patient consultation and student education.
In training, mixed reality glasses are helpful when a surgeon really needs to perceive 3D depth, like for some robotic surgery scenarios, or the position relative to the patient for procedures such as cardiopulmonary resuscitation (CPR) or intubation, explains Glassenberg. For surgical planning, high-tech glasses can aid understanding of the relative position of things, such as different anatomical landmarks. And during surgery on real patients, mixed reality glasses can be potentially helpful by projecting a digital twin so a surgeon can see magnetic resonance imaging (MRI) or computed tomography (CT) data projected onto a patient as they move around and perform the procedure.
Crucially some of the same experience can be delivered without headsets. Level Ex’s applicationswork with or without external gear and can work over video conferencing software like Zoom or a mobile phone network, says Glassenberg, a gaming industry veteran who worked for both LucasFilm and Microsoft’s gaming divisions before launching Level Ex in 2015.
The introduction of Cloud-based immersive technology is a game-changer, he says. In the U.S. alone there is an army of over 100,000 sales reps and trainers who travel from hospital to hospital to train doctors. Instead of having a sales rep explain a product’s benefits with slides, Level Ex’s immersive technology allows doctors and the sales reps to do the procedure together – trying the procedure with and without the new tool – to play with it,” says Glassenberg. “It is much better than training on a live patient,” he says.
Level- Ex, which has over 750,000 medical professional players, is also helping doctors keep up with medical advances. It is the only gaming application that has been approved for Continuing Medical Education (CME) certification, which is required every two years for practicing physicians, says Glassenberg.
“Ultimately what we are trying to do is to help doctors develop new skills,” he says. “Medical training hasn’t changed much in 50 years. It is largely based on lectures and Power Point slides. The best way to learn something is to play with it. For three decades the video game industry has been using challenges and rewards to trigger the neurochemicals for driving human learning. We do the same to help doctors learn new skills, to make mental models of complex systems, such as how drugs interact. We have games that help doctors remember clinical study guidelines, puzzle games that focus on reductive reasoning to help them make better diagnoses and strategy games to help them manage complex patients over time.”
The games can be used not only to improve skills but also to solve societal problems.
For example, studies in the U.S. show that patients of color have worse outcomes with dermatologists because these patients make up less than 10% of skin doctors’ patients. Doctors are not used to spotting rare diseases on darker skin tones, say Glassenberg. Level Ex wanted to create a game to improve the odds by pulling in images from literature for doctors to study, but the images didn’t exist, he says. To solve the problem the company built a system on game technology that can recreate skin diseases on skin of any color in any lighting environment. “This is an example of how doctors can train their brains and patients can benefit from the outcomes,” says Glassenberg.
In Level Ex’s vision of the future by 2027 gaming will become the norm in medical marketing and training. Games will be used for everything from how to diagnose disease to understanding complex mechanisms of a drug or dealing with complex surgeries, says Glassenberg. Some 80% of physicians will play immersive medical games at least once a year and 30% will play these medical games once a month, he predicts.
How Other Sectors Might Use Immersive Technologies
Glassenberg says he believes that in the near future professional games will become its own category on app stores. “Gaming will become relevant in other disciplines such as high-end manufacturing, automotive, oil and gas and any industry that involves scenarios where you need to develop mental models of complex systems,” he says.
Examples include simulating billions of molecular interactions to see how they react. “This is interesting not just for the medical profession but also for anyone in chemical manufacturing,” says Glassenberg.
“In the future augmented reality and gaming will be a key approach to learning and testing,” he says. “The technology will allow professionals to jump into a digital environment and collaboratively manipulate digital twins before trying it in the real world.”
This article is content that would normally only be available to subscribers. Sign up for a four-week free trial to see what you have been missing
