Home » Blog » Tech Trends » Training the Next Generation of Surgeons with Augmented Reality

Training the Next Generation of Surgeons with Augmented Reality


Two-thirds of the world’s population don’t have access to safe surgery. That’s as many as 5 billion people.

And of the 313 million procedures undertaken globally every year, only 6% occur in the poorest areas where over one third of the world’s population lives.

To cope with this huge problem, it is estimated that by 2030 we will need to train over 1.27 million additional surgical providers, which of course presents some enormous logistical and financial challenges.

Proximie 1

Proximie is a start-up looking to solve this problem with an Internet-enabled software tool that allows doctors to guide, help, support and train surgeons through the use of Augmented Reality. According to its Co-founder Dr. Nadine Hachach-Haram, the key advantage it offers is that it’s much more interactive than traditional simulation or live streaming tools, thus enabling higher levels of collaboration and better learning outcomes.

Two thirds of the world’s population don’t have access to safe surgery Share on X

The Augmented environment is created by overlaying two live video feeds, one in the surgical theatre filming the operation, the other pointing to a blank surface connected to the surgeon’s tablet or laptop. This allows the surgeon to draw on their screen while looking at the live surgery feed, and for the remote team to see the marks made by the surgeon and follow them precisely at their end.

Surgery is a very visual so it’s far more useful, interesting and engaging to watch it live Share on X

They are one of the companies being supported in their rapid growth by the UK Lebanon Tech Hub and have been attracting significant praise and attention from both the scientific and technology communities, winning the prestigious Royal Society of Medicine prize for Innovation in e-Health and being invited to join the MIT “Arab World Meets Silicon Valley” forum at TechCrunch Disrupt in September.

Dr. Hachach-Haram, a Lebanese surgeon based in the UK is passionate about the use of the software as a training aid for students:

When we had the old-fashioned viewing galleries, dozens of students were able to watch surgery without affecting or crowding the surgical area, and could engage with the surgeon and ask questions,” She recalls. “Fast-forward to 2016, where we have much smaller operating theatres, reduced facilities and overstretched operating lists- this environment is much less amenable to surgical education and engagement from medical students.

So Proximie is effectively updating that viewing gallery experience using 21st Century technology, so that a large number of students can log in and watch a procedure from wherever they are in the world.

Surgery is a very visual, precise and anatomical part of medicine and no matter how many times one reads it in a book, it’s always far more useful, interesting and engaging to be able to watch it live, ask questions, and visualise anatomy and pathology.

Proximie 3


In September they will launch a pilot in partnership with UCL’s Royal Free Hospital to integrate Proximie into their surgical curriculum. The educational version of the platform also has a student dashboard function, which allows them to access elements relevant to their educational curriculum, giving the ability to take notes and screenshots for later review, for example.

A cohort of students will be given access to the platform and will be able to log in to a number of operating theatres to observe the surgeries in an interactive way with the added elements of annotation, AR and other great features that will make this a completely new way of learning.

This cutting -edge technology (no pun intended) includes features such as the ability to annotate live streams and medical records integration, meaning doctors can easily access information about a patient during surgery and take snapshots during the procedure which are automatically added to that patient’s records.

Proximie 4


Proximie is hardware agnostic and requires minimal infrastructure, working with any tablet, smartphone or laptop and providing an intuitive interface with no need for expensive tools. Surgeons can literally use AR to draw on their screen using something as simple and affordable as a capped bic pen.

Proximie's cutting -edge technology includes the ability to annotate live streams and medical records integration Share on X

This technology could have a significant impact in the quality of care provided to patients in conflict zones such as Gaza and Syria. The software allows an experienced surgeon to be virtually “parachuted in” to support and guide a procedure with their specialist expertise, without any of the danger and cost of physical travel.

There are, of course, some logistic challenges associated with live streaming technology, as anybody who’s ever been on a pixelated Skype call knows. Dr Hachach-Haram says that in situations where in remote areas or conflict zones they have modified the application so that it will still provide high quality video stream even at lower network speeds. “It’s an all-round adaptive solution, and while we’re a commercial platform, we’re also committed to social impact projects, as that’s a core component of our DNA. Proximie offers free licences to charities to support them in tackling inequality in surgical knowledge and expertise.”

Currently the telesurgery feature needs to be scheduled in advance, but future iterations are already planned where the service will host a database of surgeons all over the world listed according to expertise, allowing colleagues to check their availability and seek assistance and advice in real time.

Even in developed countries such as the UK, the potential for this type of technology to maximize efficiency is enormous, as surgeons would be able to support procedures in different hospitals without the time and costs associated with commuting to various locations. It would also ensure better patient care, as the best specialist for a particular procedure could support it regardless of their geographical location at any given time.

Even in countries such as the UK, the potential for this technology to maximise efficiency is enormous Share on X

Dr. Hachach-Haram says they will be watching the developments in the Virtual Reality market with interest, and it is not difficult to imagine how, as VR technology and hardware become more accessible and pervasive, it will offer even greater possibilities for immersive learning and expertise sharing.

Virtual and Augmented Reality certainly represent a booming sector, and there is a growing interest in the medical applications for these technologies. According to a recent report from Frost & Sullivan the next 10 years are set to redefine the future of both business and consumer processes and interactions and will be adopted widely across industries by 2025.

“AR and VR will be a giant step in transforming and democratising education and businesses, especially in medicine, where immersive technology can facilitate education and access-at-a-distance for medical care,” said Dr Rafael Grossmann, who was the first person to stream a live surgical procedure with Google Glass, and his company Medical Realities, led by colleague Dr Shafi Ahmed, live-streamed the first VR surgery from The Royal London Hospital, with 54,000 viewers virtually present in the operating theatre.

From all around the world, they gained an exclusive point-of-view demonstration and interacted in real-time with the surgical event. Imagine how this level of interaction can deliver next-generation education for students worldwide, especially in developing countries, added Dr Grossmann.


There are also interesting developments in the hardware arena, with Georgian start-up InGlove developing a VR glove which could prove useful for applications such as medical simulations, and Purdue University researching a system called DeepHand, which uses a “convolutional neural network” that mimics the human brain and is capable of “deep learning” to understand the hand’s nearly endless complexity of joint angles and contortions. It uses a depth-sensing camera to capture the user’s hand and figure out where the hands and fingers are, applying algorithms to then replicate those motions in real time. Karthik Ramani, director of the University’s C Design Lab says that in Virtual Reality it is crucial for your hands to be able to accurately interact with the virtual world in order to provide full immersion.

Imagine how this level of interaction can deliver next-generation education for students worldwide Share on X


This series of photos shows the use of a new system, called DeepHand, developed by researchers in Purdue University’s C Design Lab. It is capable of “deep learning” to understand the hand’s nearly endless complexity of joint angles and contortions, an advance that will be needed for future systems that allow people to interact with virtual environments in virtual and augmented reality. (Purdue University image/C Design Lab)

It will be interesting to see how immersive tech develops and find new applications across areas such as medicine and education, and many agree that this will be the crucial factor in determining how quickly VR becomes pervasive.

For companies looking to get into VR/AR/MR our Virtual Reality Consultancy services offer guidance on how these technologies can enhance and support your brand strategy.

Alice Bonasio is a VR Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio and @techtrends_tech on Twitter.