How medical visualization tools enhance learning by reducing the cognitive load on our brains.

Immersive learning technologies are proliferating fast and also growing in popularity, being used not only for entertainment but also in a variety of professional applications.

immersive learning tools provide superior capabilities because they reduce the cognitive load on the learner, and more accurately represent these systems as they truly are
Share on X

Some of the most exciting applications of immersive technology have come in the form of training applications, and the medical sector has been fast to embrace this innovation, recognizing their great potential to improve the quality and quantity of training epochs, reduce training costs and to enhance patient satisfaction. A win-win-win.

Immersive learning technologies are proliferating fast and growing in popularity
Share on X

Broadly speaking, these technologies come in two forms: Augmented Reality (AR) – in which digital information is overlaid onto the learner’s field of view – and Virtual Reality (VR) in which the learner is immersed in a completely new environment.

Some of the most exciting applications of immersive technology have come in the form of training applications
Share on X

In a previous post I discussed the implications of xR for broad-based medical training. In this article, I will focus on the problems surrounding medical visualization. Namely, how do learners best understand and mentally represent complex systems like the brain or the human body? I offer a psychological and brain science perspective and argue that immersive learning tools provide superior capabilities because they reduce the cognitive load on the learner, and more accurately represent these systems as they truly are. If done correctly, this will significantly enhance learning.

Let’s look at the challenges medical students face when learning about the anatomy and physiology of the human body. The human body is a 3-dimensional structure that functions as a dynamic system. The ultimate goal of all medical visualization training is to facilitate the formation of a 3D dynamic mental representation of the human body in the learner’s brain that perfectly mimics the actual form. The best way to achieve this goal is to present the learner with a 3D dynamic visualization tool that perfectly mimics the human form. Unfortunately, most traditional medical visualization tools are textbooks or slide shows that are filled with 2D static images. Thus, the learner must convert a series of 2D static images into a 3D dynamic mental representation in the brain that accurately reflects the human form.

Constructing a 3D dynamic representation from a series of 2D static images requires a huge amount of cognitive effort
Share on X

From a cognitive science perspective, constructing a 3D dynamic representation from a series of 2D static images requires a huge amount of cognitive effort. First, you have to hold a mental representation of a series of 2D static images in your short-term (working) memory. Second, you have to hold these 2D static mental representations in your working memory and combine them on the fly to construct an accurate 3D static representation. Finally, you have to infer and impart the dynamic nature of the human form onto this 3D static representation.

Traditional medical training requires an enormous amount of cognitive capacity - working memory - and cognitive energy - executive attention - taxing them in such a way makes mistakes much more likely
Share on X

The cognitive science research is clear that each of the steps above requires an enormous amount of cognitive capacity (in the form of working memory) as well as cognitive energy (in the form of executive attention). Any time working memory load and executive attentional demands are taxed in such a way, you are more likely to make an error and generate an inferior mental representation. Or simply fail to understand and absorb crucial information.

Well-constructed immersive learning tools such as the Microsoft HoloLens bypass this fundamental problem by generating a highly accurate 3D dynamic representation of this 3D dynamic human form
Share on X

Well-constructed immersive learning tools such as the Microsoft HoloLens bypass this fundamental problem by generating a highly accurate 3D dynamic representation of this 3D dynamic human form, displaying it directly and accurately within the product and presenting it ready for the learner to study.

Surgeons at Imperial College London are using Mixed Reality technology to spot key blood vessels, bones and muscles prior to surgery
Share on X

This training is intuitive because it reduces the extensive working memory and executive attention demands associated with traditional medical visualization training that we outlined above. By removing the need to construct a 3D dynamic mental representation from a series of 2D static images, the working memory and executive attention load on the learner has been slashed. Equally – if not more importantly – the dynamic 3D mental representation in the learner is given a boost by the visual learning and representation systems in the brain that has evolved for just this type of learning. This system builds a 3D mental representation of objects from the dynamic flow of information on the retina.

This training is intuitive because it reduces the extensive working memory and executive attention demands associated with traditional medical visualization training
Share on X

A number of offerings of this sort exist. One that is representative is a recent application of the HoloLens mixed reality platform that is being used at the Imperial College London to help surgeons “see inside” patients before they operate. Surgeons are using this technology to spot key blood vessels, bones and muscles prior to surgery. This enhances preparation and increases the success rate of surgery. Dr. Philip Pratt, a Research Fellow in the Department of Surgery and Cancer at Imperial College London, said HoloLens is allowing surgeons to quickly and accurately understand a patient’s unique anatomy.

“Mixed reality offers a new way to find these blood vessels accurately and quickly by overlaying scan images onto the patient during the operation, explains Dr Pratt. “To perform the best operation, you have to plan it meticulously beforehand. This technology allows us to experience the data that we have collected from patients before their operation in the most realistic and natural way. You look at the leg and essentially see inside of it; you see the bones and the course of the blood vessels.”

This same logic applies to visualizing the brain, a neuron, or even a molecule, so I fully expect immersive visualization training to replace traditional training within the next five years
Share on X

The psychological and brain science research outlined above suggests that immersive learning tools have great potential in medical visualization training. This same logic applies to visualizing the brain, a neuron, or even a molecule, so I fully expect immersive visualization training to replace traditional training, and cadaverless training for non-surgeons to be the rule, not the exception, within the next five years.

The technology development often outpaces research, which why collaboration between technology developers like Microsoft and institutions such as Imperial and Case Western are crucial
Share on X

That said, efficacy/validation studies should always be conducted to verify that a particular immersive visualization training tool is superior to traditional training approaches, and should be used during the development process to make the product better. Although initial evidence and response from, teachers, learners and various experts in these fields is extremely encouraging, there is still a need for rigorous academic studies that directly and rigorously compare an AR/VR training platform with a traditional training platform on standardized tests of anatomy and physiology, as well as other critical factors known to affect learning, such as non-standard environmental conditions. It is early days, and the technology often advances at such speed that it outpaces such research, which is why collaboration between technology developers such as Microsoft and institutions such as Imperial and Case Western are highly welcome in validating the value and viability of these applications.

For companies looking to get into Immersive technologies such as VR/AR/MR/XR our Virtual Reality Consultancy services offer guidance and support on how best to incorporate these into your brand strategy.

Todd Maddox is Science, Sports and Training Correspondent at Tech Trends, and the CEO of Cognitive Design and Statistical Consulting. Follow him on Twitter @wtoddmaddox

The post Extending Reality to the Operating Room appeared first on Tech Trends.

A Silicon Valley museum is using Augmented Reality to put a tech spin on the Body Worlds concept.

Earlier this month the Tech Museum of Innovation opened a groundbreaking new exhibit to the public. Body Worlds Decoded was unveiled on October 15^th, 2017 and is set for an unprecedented 10-year run featuring bodies and specimens preserved by Dr. Gunther von Hagens’ Institute for Plastination, which created the iconic Body Worlds exhibitions to help people better understand the human body and its functions.

This is a tech twist on the iconic Body Worlds concept
Share on X

The experience uses Augmented Reality and other emerging technologies
Share on X

Funded by Silicon Valley philanthropists Ann and John Doerr, the experience uses Augmented Reality and other emerging technologies to put a Silicon Valley spin on the Body Worlds concept. In the exhibit, displays of real human bodies — eight full-body plastinates and more than 60 individual specimens —help visitors explore anatomy in a whole new way, from the smallest organs to the most complex systems, including nervous, respiratory, cardiovascular, urinary, reproductive, digestive and locomotive.

“Body Worlds Decoded is an experience like no other, and it is only fitting that this blending of nature and technology should be celebrated in Silicon Valley,” said John Doerr, the chairman of Kleiner Perkins who with his wife Ann is contributing $5 million to The Tech to make the exhibit possible. “Ann and I are thrilled to help bring this experience to life, and it is our dream that it will inspire youth and contribute to a greater understanding of the life sciences.”

In partnership with the Institute for the Future (IFTF ), the San Jose-based museum created a custom AR system called Iris that allows visitors to view and interact with immersive graphics, 3D models and virtual objects including a heart, eyeball, digestive tract and skull.

In partnership with the Institute for the Future the museum created a custom AR system
Share on X

Much like some of the anatomy applications already being deployed on devices such as the Microsoft HoloLens, this technology allows users to walk all the way around a beating heart as if it were floating in front of them, or wander around inside an eyeball, getting up-close and personal with an optical nerve if they so wish.

You can walk all the way around a beating heart
Share on X

Iris uses ARtifactor — content authoring and management software developed by IFTF — and runs on the Lenovo Phab 2 Pro, a Google Tango-enabled mobile device. The museum also plans on adding further content such as 3D models and animations to the 5,000-square-foot exhibition, so visitors will always have something new to discover.

AR will transform how we interpret our world and approach education
Share on X

“This is the future of museums —  AR is about to transform how we interpret our world and how we approach education,” said Toshi Anders Hoo, lead AR consultant and director of the Institute for the Future’s Emerging Media Lab. “The Tech has emerged as a leader in the exciting AR movement, offering a whole new world of immersive technology that will influence museums and exhibitions globally.”

The museum is aiming for Body Worlds Decoded to become Northern California’s premiere public anatomy lab as well as an AR testbed, where educators and doctors will be able to bring students and patients to broaden their understanding of the human anatomy and its functions. The AR industry will also be invited to use the exhibition to prototype advances in software and hardware with the community.

“Body Worlds Decoded is one of the most ambitious and exciting anatomy experiences ever created,” said Tim Ritchie, president and CEO of The Tech Museum of Innovation. “The human body contains so many mysteries, and the implementation of AR and all of its capabilities stands to provide valuable clues in unlocking those secrets and inspiring the next generation of advances in medicine and physiology.”

People of all ages can participate in live demonstrations of Anatomage, a 3D virtual dissection used at the nation’s top medical schools. Augmented reality is, in fact, being increasingly used to guide and train doctors and even surgeons on complex procedures.

You can participate in live demonstrations of Anatomage, a 3D virtual dissection tool
Share on X

The exhibit also explores the artistic side of anatomy
Share on X

But the exhibit goes even further and – true to the vision of the original Body Worlds shows – also explores the artistic side of anatomy, this time through installations of art inspired by the wonder and mystery of the human body. Pieces by local artist Lauren A. Toomer will be displayed, and visitors will be invited to create their own artwork.

“The human form has intrigued artists throughout history,” said Lisa Incatasciato, Exhibit Content Developer at The Tech. “Despite technology giving us a more accurate look, there’s still something mysterious and inspiring about creating your own visual interpretation of the body. You don’t have to love anatomy to enjoy this exhibit, but we bet you’ll appreciate it before you leave.”

If you would like to find out how to leverage VR/AR/MR in your own project or enterprise, Tech Trends offers Virtual Reality Consultancy support

Alice Bonasio is a VR Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio and @techtrends_tech on Twitter.

The post Augmenting Anatomy appeared first on Tech Trends.