Alice Bonasio VR Consultancy MR Consultancy Tom Atkinson Tech Trends News Reviews AR MR Mixed Reality Virtual Augmented Education Sex IOT ED Health Fin Finance Food
Home » Blog » Tech Trends » Changing the Way Surgeons Operate with Mixed Reality

Changing the Way Surgeons Operate with Mixed Reality

posted in: HEALTH Tech, Tech Trends, VR Tech


Healthcare is one of the areas with most potential for dramatic improvement with the implementation of Virtual, Augmented and Mixed Reality Technologies

The virtual world is already having a real impact in people's lives and health Click To Tweet

As countries around the world struggle to find health care provision models that balance the needs of aging populations with shrinking budgets—and as startups jostle to get into operating rooms—the virtual world is already being used to impact the health care of real people.

“I could be in Cleveland and teach a group of students in, say, London with all of us able to see one another and the holograms simultaneously,” says Professor Mark Griswold, Case Western Reserve’s faculty leader for the university’s efforts with Microsoft’s HoloLens, the $3,000 developer-edition-only augmented reality glasses. “The professor can see how students are interacting with the hologram in real time, and respond immediately with additional explanations or encouragement as needed.”

Devices like the HoloLens and consumer VR helmets like Samsung’s Gear VR and Facebook’s Oculus have received the most attention as vehicles for escape, but their real-world applications are growing fastest in the workplace. IDC, which pegs current industry revenues at around $5.6 billion, says much of the growth in shipments of VR and AR headsets over the next five years will come from industrial uses (80% a year), versus consumer uses (50%), like video games, films, porn, and other entertainment. Among current non-health care HoloLens customers, for instance, are companies like Lowe’s, Volvo, and ThyssenKrupp, whose technicians use the glasses to operate on ailing elevators.

Healthcare revenues for VR and AR technologies reached nearly a billion dollars in 2016 Click To Tweet

In health care, revenues for VR and AR technologies reached nearly a billion dollars in 2016, according to Kalorama Research. Some estimate that by 2025, that number could reach over $5 billion, thanks to uses in areas like telemedicine, pain relief, robotic surgery, and, increasingly, medical simulations.

VR and AR are already being used in areas like pain relief, robotic surgery and telemedicine Click To Tweet

Medical instructors say a VR helmet, coupled with haptic-feedback “syringes,” can help a surgeon practice a complex operation in detail before carrying it out—or help a doctor with limited access to education locally get better medical training, improving patient experience and outcomes. A pair of augmented reality goggles can put an animated “patient” in front of students, making the expensive dummies obsolete altogether. And as medical operations become more sophisticated and high-tech, computer glasses could help get practitioners up to speed faster.

The United States is facing a shortfall of 100,000 physicians by 2030 Click To Tweet

Health care experts have proposed new technologies like these as one of various solutions to what some have called a crisis in medicine: The United States could be facing a shortfall of between 48,000 and 100,000 physicians by 2030, according to the Association of American Medical Colleges. Since it takes between five and 10 years on average to train a new physician, medical industry experts say the U.S. urgently needs more people to enter that training pipeline now, particularly in highly specialized fields: The greatest shortfall, on a percentage basis, will be in the demand for surgeons, especially those who treat cancer and other diseases more common to older people.

Using Mixed Reality simulations to train surgeons could alleviate the shortfall and bring in cost savings Click To Tweet

Alleviating that shortfall and upgrading decades-old simulations with mixed reality could also ferry in cost savings that can’t come soon enough. In the U.S., the cost of health care continues to surge, far beyond the price of drugs: Open-heart surgery is 70% more than the next highest country; an appendectomy over two times more. And the price for a day in the hospital is about five times more in the U.S. than other developed countries.

Better Learning Through Virtuality?

For years, surgeons have relied on 3D modeling on computers to plan complex procedures down to the millimeter, so there are as few surprises as possible. The technology proved valuable for the team that separated conjoined twins Erika and Eva Sandoval at Stanford’s Lucile Packard Children’s Hospital in Palo Alto in 2013. In that case, surgeons donned 3D glasses to study digital renderings of the twins’ organs, allowing them to perform a heart valve replacement using an incision less than half the normal size. More recently, 3D modeling has merged with VR at Stanford Health Care, where an app called True 3D technology, developed with Mountain View-based company EchoPixel, promises to increase a surgeon’s ability to visualize and plan complex procedures beforehand

“Our biggest problem is cutting into an artery or vein that we did not expect,” says Eric Wickstrom, a professor of biochemistry and molecular biology at the medical school at Thomas Jefferson University in Philadelphia, who co-authored a 2013 study on the use of 3D models in surgery.

A clinical study published last year in the Journal of Neurosurgery looked at how surgeons rehearsed their operations using a VR-based brain modeling platform developed by the company Surgical Theater. The tool, in use at New York University, University Hospitals in Cleveland and Mount Sinai among others, appeared to help surgeons reduce the time it took to repair aneurysms, suggesting it also made the surgeries safer.

15 minutes with the HoloLens could save dozens of hours in the lab Click To Tweet

Making Virtual Bodies Feel Real And Real Bodies More Virtual

The next step in the equation will be to accurately reproduce not only the look but the feel of any surgical procedure. HoloLens partner FundamentalVR is already working on the addition of haptic feedback to surgery simulations, says Richard Vincent, the company’s founder. The London-based startup is developing a tool called FeelRealVR, which it describes as a “flight simulator” for surgery. In its current version, students use a stylus in place of a syringe, which provides realistic levels of pressure and resistance in relation to a hologram of a patient’s open knee joint, for instance.

Vincent argues that tools like this can make for better learners. The enhanced cognitive involvement that comes from not only seeing, but interacting with the holograms triggers active “Involved Learning,” he says, a recognized teaching methodology in which students have been found to retain much more of the subject matter than with traditional approaches.

Surgeons rely on 3D modeling on computers to plan complex procedures down to the millimeter, so there are as few surprises as possible Click To Tweet

AR is also augmenting instruction during real-life surgeries Click To Tweet

AR is also augmenting instruction during real-life surgeries. Hands-on, face-to-face lessons during real-life operations are in short supply, often limited to the limited number of students who can fit inside an operating theater. Surgeons who can teach are in such high demand, meanwhile, that experiential learning can be increasingly infrequent at medical schools, taught only once to a small group.

“Surgery is very visual. You can read it in a book if you want but it’s not the same as watching it live,” says Dr. Nadine Hachach-Haram, a NHS registrar in plastic surgery at the Chelsea and Westminster Hospital. “Yet it’s physically difficult to get many medical students in the operating room at any time.”

Hachach-Haram is cofounder of a company called Proximie that developed a way to use augmented reality to let a distant surgeon virtually place his or her “hands” or instruments onto a patient’s body. The idea is to let experienced practitioners guide operating teams on where each incision should be made and how to proceed. After being selected by the U.K. Lebanon Tech Hub Accelerator, Proximie is now rolling out a training pilot at The Royal Free Hospital, allowing 150 of its students to log in remotely to watch surgery through the application.

Mixed reality is also improving operating rooms in less obvious ways. ByDesign, a HoloLens app, helps surgeons, nurses, and technicians save precious time in configuring the setup of operating theaters. Whenever there’s a rotation, operating rooms need to be carefully reconfigured to meet each team’s very specific requirements, since even minute errors in that context can have dangerous consequences for both patients and practitioners.

Two surgeons hundreds of miles apart can both stand in different rooms looking at accurate holographic renditions of the same equipment Click To Tweet

Traditionally that means multiple people moving around heavy, delicate, and expensive equipment to test various configurations. In an environment where most facilities already operate near capacity, this resource-intensive process translates into higher costs and slower delivery of care, says Andy Pierce, president of Global Endoscopy at Stryker.

By donning an AR headset, practitioners can visualize objects in full-scale 3D, with the flexibility of being able to easily move virtual objects. Two surgeons hundreds of miles apart can both stand in different rooms looking at accurate holographic renditions of the same equipment, moving them around until they’re satisfied the optimum setup has been reached. This can then be saved and relayed to those in charge of setting up the operating rooms themselves.

Upgrading The Cadaver

A review of augmented reality in medical training published last year in the journal Surgical Endoscopy couldn’t say whether the technology would contribute to patient safety. But author Esther Barsom, a researcher in the department of surgery at the University of Amsterdam, noted that AR is “preeminently suitable” for helping improve the training of situational awareness during operations, a facet that is “lacking in medical curricula.” And, she wrote, “as training methods become more engaging and reliable, learning curves may be expected to become steeper and patients will ultimately benefit.”

Prof. Wickstrom of Thomas Jefferson University suggests that virtual simulations could also widen the recruiting pipeline: By making medical education more interactive and engaging, the medical field could become more accessible and attractive to those who previously might have been put off by traditional medical learning.

As training methods become more engaging and reliable, learning curves may be expected to become steeper and patients will ultimately benefit Click To Tweet

The time-honored way of teaching anatomy, for instance, is to have students spend months dissecting cadavers. But these procedures—not unlike the realistic mannequins at SimLearn—can cost tens of thousands of dollars. And cadavers can only be used by a limited number of students, and, naturally, only once.

Under Prof. Griswold at Case Western Reserve, anatomy students use the HoloLens to interact with virtual patients and organs, allowing them to pull back the various layers of the human body, visualize the muscles on top of the skeleton, and understand exactly where things are located. AR means that medical students are not confined to learning the terrain of a single body, but can see accurate visualizations of particular conditions such as cancers, heart disease, or spinal injuries.

“In the fall we did a pilot test of HoloLens with medical students who already had studied the cardiothoracic region for several weeks in the cadaver lab,”Griswold says. “After one session viewing the same area of the body wearing the devices, 85% said they had learned something new.”

Just 15 minutes with the HoloLens could have saved them dozens of hours in the lab, students who participated in the pilot told Pamela Davis, dean of the school of medicine. Case Western is now in the process of developing a broader holographic anatomy curriculum. “The quicker our students learn facts like these, the more time they have to think with them,” Davis said at last year’s Microsoft’s Build conference. “We are teaching them to think like a doctor.”

Originally published on the Fast Company website

Tech Trends’  Virtual Reality Consultancy services offers support for companies looking to get a foothold in the growing Mixed Reality spectrum and enhance their brand strategy with these exciting new technologies. 

Alice Bonasio is a VR and Digital Transformation Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio on Twitter.