Removing brain tumors is a difficult, intricate task but soon surgeons’ skills will be augmented by the ability to operate while simultaneously seeing an image — magically floating in the air — of the inside of the patient’s head.
Adding this new dimension to surgical procedures and medical imaging is the objective of a partnership between U.S.-based Magic Leap, which makes augmented reality (AR) technology that allows digital objects to be interspersed with the real world, and Germany’s Brainlab, a medical technology company that already powers treatments in radiosurgery as well as numerous surgical fields including neurosurgery, orthopedic, spine and trauma, in thousands of hospitals worldwide.
This futuristic operating system will combine Brainlab’s data management, cloud computing, visualization and data pre-processing software with Magic Leap’s spatial computing and experiential platforms.
The first release of the product promises to enable surgical planning and simulation in an office setting., The companies say this will be expanded upon quickly to include a next-generation mix of virtual and physical worlds for the operating room, radiotherapy treatment room, intensive care unit and radiology suite.
The partnership with Brainlab represents a pivot for Magic Leap, which has raised an eye popping $2.3 billion in funding. It initially targeted consumers and entertainment applications such as gaming and launched its first AR headsets , which sell for $2,295, last August. While consumer applications are still part of the mix the company is now additionally looking at how the technology can be used as a productivity tool for business, says Rio Caraeff, Magic Leap’s Chief Content oOfficer.
“ We are looking at the industries that are already working in 3D to do things like plotting a surgical map of the brain or designing a hospital or the newest sneaker,” he says. “There are already many industries and customers that speak the lingua franca of the medium because this is already in their work flow but they have to go through the awkward process of working on a 2D screen, taking it to 3D and then going back into 2D. The work flow is suboptimal so we have a receptive audience in all of these industries. This is the lowest hanging fruit but we are in the dawn of something new right now.”
The something new is using light to render digital in such a way that the brain believes it to be real. The Magic Leap One device also uses sensing to see the physical world so that digital content can respect, recognize or interact with physical surfaces and objects, preventing, for example, a digital personna from walking through a real-world desk.
Merging The Physical And The Digital
“ Any physical objects that you use in your home or at work are a collection of atoms. What happens if the atoms go away and the bits are freed and can mix with the photons surrounding us in the real world?” asks Caraeff. “This requires a leap in imagination for people who don’t come from the world of AR but Magic Leap has found a large arena of customers who see real opportunities and are trying to solve real problems with spatial computing today. We will still carry the flame for consumers but we need to go where we have customers and opportunities.”
The sectors Magic Leap is targeting include architecture, design, manufacturing and medical imaging and healthcare. “We are starting to talk to CIOs of Fortune 500 companies and .people who have tremendous expertise but are limited by legacy computing paradigms,” he says.
This new field of spatial computing will allow people to work in new ways, using telepresence and teamwork. It will, for example, permit an experience that equates to having many people in different physical locations use the same computer and same applications at the same time to collaboratively work on a 3D design or some other type of business problem.
A Potentially Powerful Diagnostics Platform
The medical field is a natural way for Magic Leap to start branching out into business applications, says Caraeff. Rony Abovitz, Magic Leap’s CEO, previously founded Mako Surgical Corp., a company that manufactures surgical robotic arm assistance platforms for operating rooms. “He has a high degree of comfort and reference for how careful you have to be,” says Caraeff. “It is not something you blindly rush into.” Magic Leap partnered with Munich-based Brainlab because “ we think it is one of the most progressive and innovatitive companies in that space and can move quickly,” he says.
The partners have not yet announced customers or a a launch date for their operating room technology. “ We are still dealing with the kind of heavy lifting you have to do before you ship,” says Carraeff. In this case that includes how to safely deal with ambient radio waves in an operating room and ensuring the technology confirms with HIPAA rules which that govern the protection of individuals’’ medical records and other personal health information.
If customers opt -in Magic Leap says it is separately looking at how it could become a powerful diagnostic platform, says Caraeff. “ Our vision is to have an SDK [a collection of software used for developing applications for a specific device or operating system] for health and wellness so that qualified developers can work on our platform.”
Tackling New Types Of Privacy Issuess
Magic Leap’s technology illustrates the benefits that intelligent connectivity will enable but, like other new innovations, also underscores the need to address how such services should be governed. “«We are amplifying the human condition : am I still I and are you still you if I am perceptively present among the people, places, and things that surround you ? » asks Caraeff. “ Magic Leap is amplifying you and giving you super powers such as telepresence and X-Ray vision,” he says. “ It is a way of being more connected to people and holds lots of promise and potential. But when you are on a platform that is so intimate in nature we have to be closely attenuated attuned to the privacy issues associated with live streaming everything you do and empower users to be in control of all of their information.”
Magic Leap says it regards privacy as one of its core values and vows to “ bake it” into its operating system, SDK and terms of use. “ «Privacy is fundamental when you are amplifying humans with sensors and unlocking the magic with a biometric scan of the retina,” Caraeff says. “ People need to know who will have access to that information. It will not be stored anywhere and will not be uploaded to the cloud. Everything you do with Magic Leap will be based on privacy by design. Our business model is not selling your data, it is making sure you have great experiences while allowing you to control your data every step of the way.”
Future Portals
Today the experience of spatcial computing — and the ability to do things that are cooperative and coordinated — requires that anyone using the technology wear clunky, expensive AR headsets. But that is set to change in future. “ The world is not going to be a homogenous place,” says Caraeff. “ « We tend to have different models of phones and people will want their devices to be interoperable with our platform so it may be that you will have mobile AR on your phone and somebody else will have it on a wearable and they will be able to communcicate, collaborate or see the dragon that lives in the park. Ensuring the future operability of shared experience is key to us,” he says. “ We don’t live in that world yet but we are building it.”
