We live in a time of amazing technological advances. From the “supercomputers” we carry in our hands, and the plethora of apps that they offer, to smart appliances and automobiles, and immersive technologies such as augmented and virtual reality. Technology is advancing at an exponential rate and has the potential to increase our efficiency and quality of life. Unfortunately, too often the primary focus is on the technology, with much less focus on optimizing the interface and the experience for the user (the UI/UX). Although computing power, specialized graphics, controllers, and the like are important, their value is diminished if they don’t engage the user in a way that is natural, and in a way that makes achieving the desired goal easy.

Too often the technological “wow” factor dominates, when what is more important is how the user’s brain and body are engaged in the interest of achieving some goal
Share on X

Unnatural UI/UX increases the likelihood of failure, and results in ineffective and underutilized technology. Too often the technological “wow” factor dominates, when what is more important is how the user’s brain and body are engaged in the interest of achieving some goal. Technology for the sake of technology is not an advance, but technology that increases our efficiency by embracing the importance of optimizing the interface with the user and the naturalness of the user’s experience, offers a huge advance. To build technologies that meet these demands one must understand how the human naturally experiences and interacts with the world.

How Humans Interact with Their World

The human body and the brain that drives it, have evolved in a world rich in context. We experience and interpret the world through our senses (e.g., sight, sound, touch, smell) and interact with and affect our world through verbal and non-verbal behavior. We build devices to help us solve problems and to perform more efficiently but all too often these devices offer context-poor experiences, and require interactions that are unnatural. This increases the likelihood of failure and decreases effectiveness. For example:

• We experience and interact with a world that is 3-dimensional, dynamic, and whose views change when we shift fixation. 2-dimensional, static worlds are unnatural and less effective.
• We experience and interact with a world that is multi-sensory (sight, sound, touch, smell) and immersive. Visually dominant, small planar worlds are unnatural and less effective.
• We experience and interact with a world where we can shift our attention seamlessly, but we always have significant awareness of our surroundings. Focused attention with practical oblivion of our surroundings is unnatural and less effective.
• We experience and interact with our world and the world interacts back through sights, sounds, touches, smells, temperatures. Thumb- or finger-only interactions are unnatural and less effective.

We must build devices that offer a context-rich user interface, and natural user experiences
Share on X

We must build devices that offer a context-rich user interface, and natural user experiences. To achieve this aim, we need to understand how the human being and its brain have evolved to experience and interact with the world, and to emulate that as much as possible when developing software and technology. One must remember that UI/UX is about the user interface and the user interaction. It is about the user. To understand the user, one must understand the user’s brain; how the brain processes information to achieve goals, and how the brain interacts with technology to achieve goals.

The Psychology and Neuroscience of Human Experience and Interaction

The human brain is comprised of at least four distinct systems that define how humans experience and interact with the world. The experiential system has evolved to represent the sensory and perceptual aspects (whether visual, auditory, tactile, or olfactory) of an experience or interaction with the world. These are the sights, sounds, touch and smells in our environment. The brain processes sensory information in distinct brain regions, but critically the perceptual representations are highly interactive, and yield a rich contextual framework for each interaction. The critical brain regions associated with the experiential system are the occipital lobes (sight), temporal lobes (sound), and parietal lobes (touch and smell).

One must remember that UI/UX is about the user interface and the user interaction. It is about the user
Share on X

The cognitive system is the information processing system. It takes sensory and perceptual information, usually in the form of text or graphics, and translates that into information that can then be acted upon. The cognitive system relies heavily on working memory and attention that are both limited capacity and form a processing “bottleneck”. One must always be wary not to overload this system as overloading will lead to errors and inefficiency. This system encompasses the prefrontal cortex and hippocampus.

Neuroscience of Human Experience and Interaction

The behavioral system in the brain has evolved to drive motor behavior. These include moving one’s arms, hands, or mouth in such a way to lift, move, type or speak. This system learns behaviors when they are followed in real-time (literally within milliseconds) by corrective feedback that rewards correct behaviors and punishes incorrect behaviors. Once learned, behavioral repertoires are elicited by context-rich environments that have been associated with particular behaviors. In short, this system learns stimulus (e.g., rich environmental context) – response (e.g. speech or motor movements) mappings. This is why certain contexts elicit behaviors without the user even “thinking” about it. This system relies on the striatum, and dopamine-mediated reward and punishment signals.

The emotion-motivation system in the brain has evolved to process positive and negative emotions (e.g., happiness, sadness, anger, frustration, regret, etc.) and motivations such as rewards and punishments
Share on X

The emotion-motivation system in the brain has evolved to process positive and negative emotions (e.g., happiness, sadness, anger, frustration, regret, etc.) and motivations such as rewards and punishments, as well as stress, pressure and anxiety. Positive emotions and motivations can enhance cognitive processing and negative emotions and motivations can dampen cognitive processing. Positive and negative emotions and motivations define the rewards and punishments that drive behavioral learning. The critical brain regions are the amygdala and other limbic structures.

Optimized UI/UX:

Narrowing the Gap Between Human Experience and Interaction with the World, and Human Experience and Interaction with Technology

One thing that is clear from the psychology and neuroscience of human experience and interaction is that it is multi-faceted. It is rich in sensory inputs and perceptual representation. It is 3-dimensional, dynamic, and immersive. Although cognitive and information processing is important, it is only one part of human experience and interaction, and it is also the most fragile because working memory and attention are limited capacity resources. One should not expect this system to do the heavy lifting without significant support. Humans are highly interactive social and behavioral machines. We interact and communicate effectively through speech and non-verbals. We also have a strong behavioral repertoire from gross motor movements like lifting, to fine motor movements like typing or threading a needle. Because our behavioral repertoire is so broad, we should leverage this as we build the UI/UX for technologies. Finally, we are emotional organisms that use emotion and motivation to drive behavior, to remain engaged, to relate to others and to persevere through hard times.

Because our behavioral repertoire is so broad, we should leverage this as we build the UI/UX for technologies
Share on X

To optimize UI/UX with technology we need to keep the psychology and neuroscience of human experience and interaction at the forefront. Every step that we take that narrows the gap between how humans experience and interact with their world and how they experience and interact with technology, is a step in the right direction. In general, technologies with the best UI/UX are those that broadly and naturally engage multiple systems in the brain in synchrony, and in a way that minimizes cognitive load, enhances engagement and motivation, all while optimizing the speed and accuracy of motor behavior.

Experiential system: Every interaction with technology is an experience. You do not want UI/UX to focus exclusively on 2-d, static, visual displays, with thumb presses, or swipes as the only form of interaction from the user, and new 2-d, static visual input as the only response from the technology. You want experiences to be rich in context. The more 3-dimensional, dynamic, and multi-sensory the better. Users should interact with technology using visual fixation (without touch), speech, and motor behaviors that use more than the thumbs. Technology should respond to users with a rich array of color, form, motion, sound, and tactile assets. These should be constructed in such a way as to optimize the user’s experience and interaction. The user needs to interact with the technology, but the technology needs to interact with the user.

You want experiences to be rich in context. The more 3-dimensional, dynamic, and multi-sensory the better. Users should interact with technology using visual fixation, speech, and motor behaviors that use more than the thumbs
Share on X

The more sensory and perceptual systems are engaged in the interest of achieving the desired goal, not simply to increase the “wow” factor, the more memorable and enjoyable the experience, and the more likely the user is to achieve the goal quickly and accurately. Technologies like augmented and virtual reality hold great promise for engaging the experiential system. Augmented reality does this by overlaying assets onto the real world. Virtual reality does this by immersing the user in a new, virtual world. Technologies that utilize speech recognition also address these issues (e.g., Alexa, Siri, Google). Technologies that utilize wearable sensors or some other method of tracking full hand and arm movements as part of the user interface will be more natural for users. In addition, the plethora of sensors that track the user’s state of readiness including fatigue, stress, and vigilance, can be utilized to more effectively interact with and guide users.

Technologies like augmented and virtual reality hold great promise for engaging the experiential system
Share on X

Cognitive system: Too much UI/UX taxes this system without providing support from other systems. Too often technology is purely visual, and requires so much attention and working memory from the user, that they are functionally oblivious of their surroundings. This is hazardous, but also means that any lapse in attention or overload of working memory will lead to failure. The goal is to reduce the cognitive load by providing support from other systems and by providing users with what they need, where they need it, and when they need it. Again, technologies like augmented and virtual reality will be useful here, as their assets can be overlaid on the real or virtual world in a way that minimizes cognitive load and provides an experiential scaffolding.

Even so, much more attention is needed to reduce cognitive load, even with traditional 2d, planer technologies. Breaking a task down into small, manageable units (much like microlearning in Learning & Development) can reduce the cognitive load, and increase the chance of success. Every incremental increase in cognitive load is an incremental increase in difficulty, likelihood of failure, and likelihood of abandoning the technology. Technologies that are effective at achieving these goals are usually the most successful. Although perfection has not been achieved, and advances are still necessary, incorporating speech, natural language processing, audio and more natural behavioral interfaces will go a long way toward addressing some of the bottleneck associated with cognitive load.

Advances are still necessary, incorporating speech, natural language processing, audio and more natural behavioral interfaces will go a long way toward addressing some of the bottleneck associated with cognitive load
Share on X

Behavioral system: Technologies that optimize the naturalness of behavioral interaction are rare. This is surprising given the large body of groundbreaking research on procedural and motor skills. With the majority of UI/UX being haptic and click- or touch-based, understanding this system is especially crucial. Three critical findings are relevant. First, a high-quality user interface builds a collection of relevant behavioral repertoires through real-time immediate feedback that will facilitate interaction with the technology today and will enhance the ease of interaction in the future. This means that appropriate behaviors should be clearly rewarded, and inappropriate behaviors should be clearly punished, in real-time. Simply moving on to the next step following a correct action is not enough. Provide a highly motivating, immediate feedback signal, and that behavior will be more likely to follow in the same context next time. Second, a high-quality user interface implements related tasks with similar motor behaviors and unrelated tasks with dissimilar motor behaviors. Motor skills can be complementary or competitive and this should be incorporated directly into the UI/UX.

Motor skills can be complementary or competitive and this should be incorporated directly into the UI/UX.
Share on X

If two interactions with a technology achieve a similar goal, then the motor behavior to achieve those goals should be similar (e.g., to accept some choice, swipe right). On the other hand, if two interactions achieve different goals, then the motor behavior should be qualitatively different (e.g., swipe right to accept and swipe left to reject). Finally, a high-quality user interface needs to use natural motor behaviors to achieve goals. If my goal is to use virtual reality to practice a medical procedure that does not involve button presses, then there should be no button presses as part of the user interface. Too many virtual reality training tools show realistic virtual objects, but the interaction is not realist and is unnatural. This trains associations between unnatural behaviors and outcomes. We need to move beyond thumb- and click-based behaviors, and beyond unnatural hand controllers. Speech and full hand gestures must be incorporated using sensors and other methods of tracking.

Emotion-motivation system: UI/UX that effectively engages the emotion-motivation system is critical. When training behavior the UI/UX must be such that behaviors are rewarded or punished in real-time. Without this behavioral learning will be suboptimal. Technologies like virtual reality are especially effective at engaging emotion and motivation centers. The ability to be transported into any situation, whether emotion-laden or not, is critical for building situational awareness that is so important in so many aspects of our lives. In addition, the ability to be transported into another person’s shoes, to see the world from multiple perspective, and to “walk a mile in someone else’s shoes” will go a long way toward building the people (aka soft) skills and empathy that are central to success and to a positive workplace and society.

UI/UX that effectively engages the emotion-motivation system is critical. When training behavior the UI/UX must be such that behaviors are rewarded or punished in real-time
Share on X

The immersive technologies of virtual and augmented reality go a long way toward addressing many of the UI/UX concerns raised above. First, and foremost, they are grounded in context-rich experiential learning that is 3-d, immersive and dynamic. This is our natural state. Immersive technologies offer the promise of minimizing cognitive load, but often there is a conflict between efficiency due to reduced cognitive load, and “flash”. Good solid iterative experimentation can address this concern. Although there is still work to do to incorporate truly natural haptics, as well as speech recognition, and real-time measures of physiology, augmented and virtual reality offer great promise as tools to train behavior. Finally, immersive technologies, especially virtual reality, can train situational awareness, people skills and empathy by effectively engaging emotion centers in the brain.

The most important take-home message, and one that continues to be missed by so many, is that technology is there for the user. The best technology conforms to the user’s natural tendencies and leverages those to achieve goals. The worst technology requires the user to conform, and expects unnatural tendencies to get the job done. The more we place the user at the center, and not the technology, the better.

Todd Maddox is Science, Sports and Training Correspondent at Tech Trends, and the CEO of Cognitive Design and Statistical Consulting. Follow him on Twitter @wtoddmaddox

The post Why is UI/UX so Important? appeared first on Tech Trends.

A newly published white paper investigates the pedagogical theory and use cases for deploying mixed reality in the classroom.

It is easy to get excited about the possibilities the Mixed Reality spectrum brings to education. Humans are essentially programmed to learn best through experience, and that is just what immersive technologies enable: we’re not just learning it, we’re living it.

Humans are essentially programmed to learn best through experience, and that is just what immersive technologies enable
Share on X

And exploring a few of those possibilities is what I had the chance to do when I collaborated with Microsoft Education to write the Immersive Experiences in Education white paper we’ve published at the ISTE 2019 conference a few weeks ago.

Methodology

Now, I’m very much a technology enthusiast. I’ve been called a futurist, evangelist, influencer, all those really annoying trendy buzzwords… But at heart I am also an educator. I worked in academia, I taught English as a second language, I’ve mentored people and for many years I worked in the EdTech sector, so I speak truthfully when I say I’m forever in awe of teachers and mindful of the importance of education in society.

Mixed Reality technology is amazing but is it really useful to educators? Does it actually influence learning outcomes, and if so, how?
Share on X

This is why when Microsoft approached me to collaborate with them in researching and writing this white paper, I wanted to take a step back and ensure that we weren’t just flashing a shiny pretty tech trinket around. The technology is amazing, there’s no doubt about that, but is it really useful to educators? Does it actually influence learning outcomes, and if so, how?

When deploying any technology in the classroom, success depends on engaging educators and building a responsive feedback loop that prioritizes learning outcomes rather than the technology itself
Share on X

Through our research and interviews, a picture of the concrete pedagogical benefits of using Mixed Reality in the classroom emerged
Share on X

I was keen to ensure that our methodology really reflected this educator-led focus as much as possible, and was happy to find that the team at Microsoft Education very much shared that vision.

The paper cross-referenced pedagogical theory with case studies obtained through personalized email interviews with teachers, students, researchers, and technologists who have successfully deployed such technology within various learning environments and contexts, and the bulk of my time was indeed spent on an extensive literature review and on gathering and processing the data from those testimonials to try and gleam insights that could be useful for educators.

I will let you judge for yourselves whether we’ve succeeded – you can read and download a free copy of the white paper here –  but our guiding principle was very much that when deploying technology in the classroom, success is dependent on engagement with educators, and on building a responsive feedback loop that prioritizes learning outcomes rather than the technology itself.

The paper cross-referenced pedagogical theory with case studies of teachers, students, researchers, and technologists who have deployed such technology
Share on X

Through our research and interviews, an overall picture of the concrete pedagogical benefits of using Mixed Reality in the classroom did start to emerge, and although further research is certainly needed and I wouldn’t dream to claim that this is the definitive document on the subject, it manages, in my humble opinion, to tease out some really important findings and a lot of interesting questions. I, for one, always think the most successful research is the one where you end up with a lot more questions than when you started, which is why I hope to hear lots of questions from the community (you can always send a tweet my way @alicebonasio or connect on LinkedIn). Below are some of our key research findings:

Immersive digital simulations allow learners to absorb knowledge within a realistic context through experiential learning

Our research indicated that the most effective educational experiences often involve mastering tasks within context of personally relevant, realistic situations. Situated learning in well-designed digital environments actually leads to the replication of those behaviours in the real world.

The key differentiator in using Mixed Reality is that it seems to dramatically increase the rate of transfer – in other words, our ability to apply and adapt learning to different real-world situations.It Mixed Reality facilitates transfer by simulating realistic scenarios within specific pedagogical contests. It literally can bring a curriculum to life for students. This means we not only can create more effective learning experiences, but also expand the range of available experiences, since we can simulate scenarios that would be too difficult, too expensive, or too dangerous for students to experience in real life.

Mixed Reality helps to prevent Cognitive overload.

This second finding probably comes as no surprise to most educators, in that trying to visualize and assimilate large amounts of complex information is HARD. It can literally trigger a bottleneck within the limited working memory of the brain, and we soon become tired of it. Attention and interest lags, and it becomes harder to engage students in the subject.

Our research indicates that the most effective educational experiences involve mastering tasks within personally relevant, realistic context
Share on X

Mixed Reality, however, can help reduce this cognitive load by allowing learners to directly visualize, manipulate, and interact with complex structures, so that the brain is not so busy trying to visualize the data, but instead engaged in understanding, assimilating, recalling, and applying it.

Test scores among students using immersive technologies improved by as much as 22%

Now, those two findings we looked at are crucial in improving learning outcomes such as test scores, increased performance of skills-based tasks, abtract reasoning, and critical thinking. It also had positive effects on retention of knowledge.

Most interestingly, however, we found indications that this technology was particularly helpful in improving learning conditions for learners with conditions such as dyslexia, autism, and ADHD.

Mixed Reality can help reduce the cognitive load on the brain by allowing learners to visualize, manipulate, and interact with complex structures
Share on X

Here, immersive technologies have been shown to assist previously low-achieving students in overcoming learning challenges to the point where there was no longer any statistical difference in test scores between them and high achieving students.

Mixed Reality 
in the classroom fosters self-directed learning

Teachers reported that using MR in the classroom promoted a more inquiry-based environment that maximized learning opportunities by instilling a greater sense of autonomy, encouraging students to creatively engage with the subject beyond the pedagogical task at hand.

“Students report a greater understanding and appreciation for the development of VR content after engaging in 3D animation and/or game design projects. In particular, students are able to see the connection between the engineering of a virtual world, and the virtual world’s role as an element of digital storytelling.” Paul Turnbull 
President Mid-Pacific Institute

Co-created content fosters a more meaningful connection by generating something in which the student has a personal emotional investment. Extended experiences that immerse students in rich contexts with strong narratives, authentic practices, and links to real-world outcomes are what truly unleash the transformational power of immersive experiences.

Co-created content fosters a more meaningful connection by generating something in which the student has a personal emotional investment
Share on X

“We led a lesson with an English class, where students designed a safe and scary space, then were able to experience them both in Virtual Reality. The lesson brought technology in the context of an English lesson by creating a spark for students to translate into their writing exercises.” Cody Karutz
Founder and CEO of Blue Trot

Like all technologies adopted within a pedagogical classroom setting, MR should remain supplementary to in-person academic programming, allowing for human relations and interactions to ultimately guide the social- emotional learning experience.For me, the biggest takeaway from this is that for Mixed Reality to be effectively implemented in the classroom we must ensure that the technology is integrated into pedagogical theory and best practice.

Immersive technologies have been shown to assist previously low-achieving students in overcoming learning challenges
Share on X

Tech Trends offers a broad range of Digital Consultancy services to guide companies, individuals and brands in effectively leveraging existing and emerging technologies in their business strategy.

Alice Bonasio is a VR and Digital Transformation Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio on Twitter.

The post Report: Immersive Experiences in Education appeared first on Tech Trends.

Contagious yawning is a well-documented phenomenon in which people – and even some non-human animals – yawn reflexively when they detect a nearby yawn. Another aspect of this is that when people are in company, the presence of others also causes most of us supress that reflex (or at least try to).

The presence of an actual person in the testing room had a more significant effect on yawning than anything in the VR environment
Share on X

Contagious yawning is a well-documented phenomenon in which people – and even some non-human animals – yawn reflexively when they detect a nearby yawn
Share on X

So when studying the psychological effects of being immersed and interacting with others in Virtual Reality, researchers at the University of British Columbia decided to use yawning patterns as a benchmark to compare people’s behaviours in real-world face-to-face interactions as opposed to VR. And as often happens with such research – specially in what is still a relatively new field – the results were mixed and somewhat surprising.

To instigate contagious yawning in a VR environment, the team from UBC, along with Andrew Gallup from State University of New York Polytechnic Institute, had test subjects wear an immersive headset and exposed them to videos of people yawning. In those conditions, the rate of contagious yawning was 38 per cent, which is in line with the typical real-life rate of 30-60 per cent.

Researchers at the University of British Columbia decided to use yawning patterns as a benchmark to compare people’s behaviours in real-world face-to-face interactions as opposed to VR
Share on X

If the gap between VR and real life could be closed, scientists would be able to examine the link between the brain, behaviour, and the human experience in both actual reality and altered realities
Share on X

However, when the researchers introduced social presence in the virtual environment, they were surprised to find it actually had limited impact on subjects’ yawning. Subjects yawned at the same rate, even while being watched by a virtual human avatar or a virtual webcam. It was an interesting paradox: stimuli that trigger contagious yawns in real life did the same in virtual reality, but stimuli that suppress yawns in real life did not.

The presence of an actual person in the testing room had a more significant effect on yawning than anything in the VR environment. Even though subjects couldn’t see or hear their company, just knowing a researcher was present was enough to diminish their yawning. Social cues in actual reality therefore appeared to dominate and supersede those in virtual reality.

“People expect VR experiences to mimic actual reality and thus induce similar forms of thought and behaviour,” said Alan Kingstone, a professor in UBC’s department of psychology and the study’s senior author. “This study shows that there’s a big separation between being in the real world, and being in a VR world.”

Which is not really surprising, as we’re of course consciously aware that what we experience in Virtual Reality is different and separate from our experience of reality itself. However, there have also been numerous studies that show how psychological effects experienced virtually do carry on to our off-world behaviour.

Yet as VR becomes an increasingly popular research tool in psychology and other fields, these findings show that researchers also need to account for its limitations, the study concludes.

When the researchers introduced social presence in the virtual environment, they were surprised to find it actually had limited impact on subjects’ yawning
Share on X

“Using VR to examine how people think and behave in real life may very well lead to conclusions that are fundamentally wrong. This has profound implications for people who hope to use VR to make accurate projections regarding future behaviours,” said Kingstone. “For example, predicting how pedestrians will behave when walking amongst driverless cars, or the decisions that pilots will make in an emergency situation. Experiences in VR may be a poor proxy for real life.”

If that gap between VR and real life could be closed, scientists would be able to examine the link between the brain, behaviour, and the human experience in both actual reality and altered realities that span place and time, Kingstone added.

We’re of course consciously aware that what we experience in Virtual Reality is different and separate from our experience of reality itself
Share on X

Scientific research like this is extremely welcome as we continue to navigate our way through these immersive technologies and our effects in our brains, but as is often the case, it leaves us with a lot more questions than answers. For example, would the use of better, more realistic avatars affect the subject’s sense of social engagement with the experience? We know that this aspect of meaningful social interaction is one of the most challenging aspects of developing the technology, which is probably why we haven’t seen a Facebook equivalent in the Social VR space emerge as yet. And all this probably means that we might be seeing a lot more exciting research about yawning in VR…

This article originally appeared on VRScout

Scientists Use Yawning To Study Social Presence In VR via @alicebonasio https://t.co/0nuI1POFix pic.twitter.com/9Lms2q9s14

— VRScout (@VRScout) January 26, 2019

As VR becomes an increasingly popular research tool in psychology and other fields, these findings show researchers need to account for Limitations
Share on X

For companies looking to get into Immersive technologies our VR Consultancy service offers comprehensive support in strategic deployment of Virtual, Augmented and Mixed Reality

Alice Bonasio is a VR Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio on Twitter.

The post Studying Social Presence in VR appeared first on Tech Trends.

Why extended Reality (xR) is an Effective Tool in Manufacturing: A Brain Science Analysis.

This report focuses on a learning science evaluation of extended reality (xR) technology’s potential as a learning and training tool in the manufacturing sector. xR technologies come in two major forms: augmented reality (AR) – in which the learner is in a real environment where digital information is overlaid onto the learner’s field of view –  and virtual reality (VR) – in which the learner is immersed in a completely new virtual environment separate from their physical surroundings.

This report shows that xR technologies hold great promise in manufacturing because they reduce the cognitive load on the learner, provide the opportunity for limitless practice, and speed learning and retention by broadly engaging multiple learning systems in the brain in synchrony.

The Importance of Learning and Training in Manufacturing

High quality training tools are necessary to enhance accuracy and efficiency in the manufacturing process, where one mistake or small inefficiency can have significant ripple effects that are costly and time-consuming. For example, individuals tasked with supply chain management must quickly and accurately evaluate inventory levels, place necessary orders, and facilitate the movement of supplies and goods. All personnel in manufacturing must be well versed on the environmental health and safety rules, regulations, and emergency action plans. This is critical for compliance purposes, but also to reduce the risk of accidents that can endanger workers and slow or stop the manufacturing process.

Experiential learning is effective because it engages multiple learning systems in the brain in synchrony
Share on X

Product development is critical and involves an iterative learning process in which prototypes are developed, evaluated, and ultimately lead to a viable product. Product manufacturing requires highly trained technicians to operate and maintain the manufacturing equipment. When the equipment breaks down operators must troubleshoot and address the problem as quickly, accurately and safely as possible. Field service is also important to keep customers happy and their equipment running smoothly.

This report focuses on a learning science evaluation of extended reality (xR) technology’s potential as a learning and training tool in the manufacturing sector
Share on X

Learning Science of Training in Manufacturing

“Learning is an experience. Everything else is just information” Albert Einstein

This is an insightful quote from Albert Einstein that is supported by learning science—the marriage of psychology and brain science. As elaborated below, experiential learning is effective because it facilitates the engagement of multiple learning systems in the brain in synchrony. Experiential learning also provides the foundation for the effectiveness of AR and VR in manufacturing and other sectors.

The human brain is comprised of at least three distinct learning systems
Share on X

The human brain is comprised of at least three distinct learning systems; a  schematic of theseis provided in the figure below. The cognitive skills learning system in the brain has evolved to obtain and process knowledge and facts. Whether comparing inventory with a checklist, studying safety rules and regulations, reading equipment training manuals, or vetting product descriptions, the cognitive skills learning system is being recruited.

Experiential learning provides the foundation for the effectiveness of AR and VR in manufacturing
Share on X

Cognitive skill learning tends to involve processing text and schematics and is limited by the learner’s working memory and attention span. It requires focus and mental repetition for long-term memory storage. The cognitive skills learning system encompasses the prefrontal cortex, hippocampus and associated medial temporal lobe structures in the brain. The ultimate goal of this system is to transfer knowledge from short term memory in the prefrontal cortex to long term memory in the hippocampus and medial temporal lobes. Processing in this system is adversely affected by stress, pressure, and anxiety. This system is slow to develop, not reaching maturity until individuals are in their 20’s, and begins to decline in middle age.

xR technologies reduce the cognitive load on the learner, providing opportunity for limitless practice and speed learning and retention by engaging multiple learning systems in the brain
Share on X

One mistake or small inefficiency can have a ripple effect throughout the manufacturing process that is costly and time-consuming
Share on X

The behavioral skills learning system in the brain has evolved to learn behaviors. It is one thing to know what to do, but it is completely different to know how to do it. Knowing the safety rules and regulations is completely different from knowing how to initiate those behaviors in an emergency. Memorizing the operations or repair manual for a piece of machinery is different from knowing how to operate or repair the machine. Behavioral skills are learned by doing.

Cognitive skill learning tends to involve processing text and schematics and is limited by the learner’s working memory and attention span
Share on X

Processing in this system is optimized when behavior is interactive and is followed in real-time (literally within milliseconds) by corrective feedback. Behaviors that are rewarded will be more likely to occur in the future, and behaviors that are punished will be less likely to occur in the future. Interestingly, this system does not rely on working memory and attention, and “overthinking it” hinders behavioral skills learning. Behavioral skill learning is mediated by the basal ganglia and gradual, incremental dopamine-mediated changes in behavior. The ultimate goal of this system is to train direct neural connections between sensory regions and motor regions in the brain that drive behavior.

As Einstein so eloquently stated, experience is at the heart of all learning. It is also the key ingredient in xR training. The experiential learning system has evolved to represent the sensory aspects of an experience, whether visual, auditory, tactile or olfactory. Every experience is unique, and adds rich context to cognitive and behavioral skills learning. The critical brain regions associated with experiential learning differ as a function of the sensory input. Visual representations are formed in the occipital lobes and auditory representations are formed in the temporal lobes. Tactile representations are formed in the parietal lobes and olfactory information is represented in the piriform cortex and olfactory bulb.

xR Applications in Manufacturing

The key ingredient of xR technology in manufacturing is that experiential learning systems are being recruited in synchrony with cognitive and/or behavioral learning systems. Here I outlined 5 use cases for xR in manufacturing.

The experiential learning system has evolved to represent the sensory aspects of an experience, whether visual, auditory, tactile or olfactory
Share on X

Supply Chain: Suppose you are tasked with inventory control. In a typical scenario, you might approach each set of inventory, count it, note this on a clipboard or tablet, and tag inventory that is low and in need of replenishment. In this case, your cognitive system is highly engaged. You must focus on the inventory and count it, then shift attention to the clipboard or tablet, find the appropriate location, write the number down, then repeating the process. The working memory and attention load are high, attention switching is necessary, and thus, the potential for error and reduced efficiency is also very high.

Every experience is unique, and adds rich context to cognitive and behavioral skills learning
Share on X

Now consider the same situation but where you don a pair of augmented reality glasses. You scan the inventory room and visual cues on the display direct you to the first set of items to be counted. You scan the items and a built-in algorithm counts the number of items present and the number needed. When the inventory is low a warning is presented and you are asked to tag this item for replenishment. The glasses then direct you to next inventory item to be counted. Once the inventory process is complete the system prompts you to send the inventory needs to the relevant purchasing departments. In this case, your cognitive load is reduced, attention switching is minimized, the experiential and cognitive systems are engaged, and you are guided through the process in the most behaviorally efficient method possible. The process is faster and more accurate and the likelihood of a supply chain error is reduced.

It is one thing to know what to do, but it is completely different to know how to do it
Share on X

Environmental Health and Safety: Suppose you work in a manufacturing plant and you need to learn the steps to take to keep the workplace safe. You could read a workplace safety manual that describes the rules and regulations with text and figures. This places a heavy load on working memory and attention to translate the abstract information into a visual representation of the steps you would need to take during a safety incident. Alternatively, you could read the manual and also watch a video that shows actual or simulated safety conditions. This is better because the experiential systems in your brain are being engaged to some degree. However, in both cases, your goal is to store this information in memory for use later during a real-world situation in which your and your co-worker’s safety is in jeopardy.

Behavioral skills are learned by doing
Share on X

Alternatively, you could be directed to predefined locations in the workplace while wearing augmented reality glasses. At each location your view of the workplace is augmented with static text overlays or dynamic video that provides specific information on the safety guidelines or usage of safety tools. For specific safety equipment, you could receive step-by-step instructions or visual labels describing exactly how the equipment works. You might then be asked to demonstrate your skill with a specific safety tool, with or without text-based prompting and receive real time feedback. Analogously, you might don a VR headset and be transported into the middle of a workplace emergency. You might scan your virtual environment watching personnel follow or fail to follow the guidelines within this chaotic situation while receiving auditory feedback on the appropriate actions.

In AR and VR, there is minimal need to translate abstract text or figures into a visual representation because this information is being supplemented with experiential learning that is proximal and salient
Share on X

In the AR and VR examples, there is minimal need to translate abstract text or figures into a visual representation because this information is being supplemented with experiential learning that is proximal and salient. Because multiple brain systems are active, the memory traces in each system are strong and interconnected. Behavioral learning will also occur when you demonstrate your skill with safety tools and receive feedback either with the AR tool, or in VR with haptic feedback incorporated.

Product Development and Prototyping: Suppose you are part of the product development team. This is an iterative process in which schematics and computer aided design (CAD) tools are utilized and ultimately lead to the development of a prototype. As with reading text and studying static figures, this places a heavy load on the cognitive skills system in the brain to construct a 3D dynamic image from a series of 2D static images. It is difficult to know if this translation is effective for each member of the team and whether all team members “visualize” the same product. This is suboptimal and inefficient.

Instead, imagine donning an AR or VR tool in which a virtual 3D dynamic prototype is presented simultaneously to all members of the product development team early in the development process. The team is either physically present in a single room or are virtually collaborating via VR. Either way, all team members can view the single prototype, can “hold” it in their hands, and can manipulate the prototype –  all while discussing specific design features that they like or dislike. In this case, the cognitive load is minimized because a 3D dynamic prototype is present from the start. In addition, because the prototype can be shared virtually, team members in any location can collaborate.

The key ingredient of xR technology in manufacturing is that experiential learning systems are being recruited in synchrony with cognitive and/or behavioral learning systems
Share on X

Equipment Operation and Repair Training: Suppose you are training on the operation and repair of a large, expensive, and rare piece of equipment. Because there are only a few training centers around the world you begin by reading and memorizing the operations and repair manuals. Then you fly to a training center for an intensive 6-week course where you attend classroom training sessions and obtain hands-on equipment training. Once the training is complete you return to work as a certified operator and repair technician. Although certified, there is still much on-the-job learning to obtain to become an expert. In this scenario, you begin the training process with a heavy load on the cognitive system as you attempt to construct 3D dynamic representations of the equipment from 2D static training manuals. With that information in hand you begin the intensive hands-on training that will facilitate behavioral skills learning. Although the practice is intense, it is impossible to obtain enough experience in a 6-week course to attain expertise.

The experiential and cognitive systems are engaged, and you are guided through the process in the most behaviorally efficient method possible
Share on X

Now suppose that you begin the training process by exploring the equipment in virtual reality. You can walk around and view the equipment while receiving verbal or written instructions about the specific operations of each part. If you have haptic hardware, you can touch the equipment and manipulate it, again being driven by verbal or written instructions. You might even have a virtual trainer in the same room with you pointing out specific features and answering questions.

Instead of flying elsewhere to a training facility, you might don your VR headset each day for virtual lessons, and your homework might involve troubleshooting an array of equipment malfunctions
Share on X

Instead of flying elsewhere to a training facility, you might don your VR headset each day for virtual lessons, and your homework might involve troubleshooting an array of equipment malfunctions. You might even be placed in a virtual emergency situation in which you receive real-time feedback or as part of a test. Once certified, you might continue with more advanced VR training on more complex and rare emergency situations, but ones that can have a major impact on the manufacturing process. In this case, you are receiving cognitive and behavioral training simultaneously with experiential learning. There is no need for physical travel, and the rare and expensive piece of equipment is available, at your fingertips, 24/7. You can train on rare and dangerous situations that would be difficult, if not impossible, to simulate during real-world training. Your confidence and level of expertise will be higher and you will achieve these more quickly.

As Einstein stated, experience is at the heart of all learning. It is also the key ingredient in xR training
Share on X

The augmented and virtual content must be optimized for human consumption
Share on X

Troubleshooting and Field Service: Suppose you are a consumer of a manufactured product (e.g., computer, appliance, DIY furniture, etc.), and you attempt to troubleshoot some problem either with online manual, YouTube videos, or on the phone with a technician. You find yourself trying to convert the abstract information into actionable behaviors to fix the problem. You are constantly switching your attention back and forth from the manual or video to your product, or trying to find the right words to explain the problem on the phone wishing that the technician could “see” what you see. This is frustrating and ineffective.

With these technologies one can quickly obtain subjective ratings of confidence, satisfaction and engagement, or develop objective tests to determine whether learning has actually occurred and to challenge the learner
Share on X

When multiple brain systems are active memory traces in each system are strong and interconnected
Share on X

Big Data: The Hidden Bonus of xR Training

The focus of this report has been on the psychological and brain-based advantages of xR in manufacturing, but there is another advantage of xR technologies that is equally important and that is the promise of actionable data. With these technologies one can quickly obtain subjective ratings of confidence, satisfaction and engagement, or develop objective tests to determine whether learning has actually occurred and to challenge the learner. These can be combined with eye gaze and heat map patterns that provide a direct window onto attentional processes and engagement, or haptic data to provide real-time behavioral feedback. These data can be used to speed iteration toward optimal solutions and to quantify the ROI associated with xR technologies.

You can train on rare and dangerous situations that would be difficult, if not impossible, to simulate during real-world training
Share on X

Quality Content and Optimized User Experiences

AR and VR technologies offer promise as effective tools for manufacturing by facilitating broad engagement of cognitive, behavioral and experiential systems in the brain that are critical in manufacturing. A note of caution though. Although these tools have great potential in manufacturing, the tool in and of itself will not meet this challenge. These tools provide augmented and virtual experiences and information to human users.

Data can be used to speed iteration toward optimal solutions and to quantify the ROI associated with xR technologies
Share on X

Thus, the augmented and virtual content and temporal presentation of the content must be optimized for human consumption. Augmented information can reduce the cognitive load, but can also overload the user. Virtual information must be realistic and lead to a veridical sense of “presence”. Optimization follows from good experimental testing and modification. Fortunately, the rich and broad set of real-time data that can be extracted from AR and VR technologies makes this a viable and realistic prospect.

Virtual information must be realistic and lead to a veridical sense of presence
Share on X

For companies looking to get into Immersive technologies our VR Consultancy service offers comprehensive support in strategic deployment of Virtual, Augmented and Mixed Reality

Todd Maddox is Science, Sports and Training Correspondent at Tech Trends, and the CEO of Cognitive Design and Statistical Consulting. Follow him on Twitter @wtoddmaddox

The post Report: How XR Works With Your Brain appeared first on Tech Trends.

In a first-of-its-kind study, Elsevier has provided comprehensive analysis of the global research landscape and future trends in Artificial Intelligence (AI)
Share on X

The report’s data shows how China is set to become the global leader in AI research, having already overtaken the U.S. in 2004 and fast closing in on Europe’s lead in that arena. At this rate, the report predicts that the country will become the biggest source of AI research globally within four years.

The report predicts that China will become the biggest source of AI research globally within the next four years
Share on X

Key findings from the report include:

• Europe is suffering from AI Brain Drain: Over the 20 year period, Europe has been losing academic talent in this area, mostly to the corporate sector in the United States.
• Globally, research in AI has grown by 12.9% annually over the last 5 years
• There have been several key milestones over 20 years which have led to spikes and growth in AI research
• The ethics of AI is a research blind spot. Despite the increasing societal relevance of AI and widespread media attention on ethical implications, research remains limited.

The Elsevier analysis finds that industry in the United States attracts the most AI talent from both local and international academia
Share on X

The report shows that, globally, AI research has accelerated, growing by more than 12 percent annually in the past five years (2013-2017), compared to less than 5 percent in the previous 5 years (2008-2012). By contrast, research output overall, globally across all subject areas, has grown by 0.8 percent annually over the past five years (2013-2017).

Enrico Motta, Professor of Knowledge Technologies at the Open University in the UK, was one of the many experts who contributed to the report. He believes it is the most comprehensive characterization of AI outputs across different sectors delivered so far: “This report applies extensive text mining and semantic analytics across literature from different sectors to uncover how to more comprehensively define the AI field – essentially using AI to map AI,” he adds.

Reviewing 600 documents and over 700 field-specific key words across four sectors – research, education, technology, and media – the semantic analysis reveals that the field of AI focuses on seven distinct research areas:

• Search and Optimization,
• Fuzzy Systems,
• Natural Language Processing and Knowledge Representation,
• Computer Vision,
• Machine Learning and Probabilistic Reasoning,
• Planning and Decision Making, and
• Neural Networks.

Globally, AI research has accelerated, growing by more than 12 percent annually in the past five years
Share on X

Of these areas, research in Machine Learning and Probabilistic Reasoning, Neural Networks, and Computer Vision show the largest volume of research output and growth.

Keyword clusters and co-occurrences in the AI field, 2017; source: Scopus

Machine Learning and Probabilistic Reasoning, Neural Networks, and Computer Vision show the largest volume of research output and growth
Share on X

Regional findings highlighted in the report:

• In terms of research areas within the field of AI, Europe is the largest and most diverse region with high levels of international collaboration
• After China and the US, India became the third largest country in terms of AI research output, followed by Germany and Japan.
• Iran is ninth in AI publication output, on par with France and Canada.
• International mobility and collaboration patterns suggest that China operates in relative isolation from the wider research community.

“With this comprehensive study of research performance in AI we aim to provide insights into the field’s dynamics, trends and parameters, says Dan Olley, Chief Technology Officer at Elsevier. “The report is not a conclusion, but the start of a discussion on how we best enter the era of AI and increasingly symbiotic technology,” he concludes.

Data used in the report comes from Elsevier’s Scopus, Fingerprint Engine, PlumX, ScienceDirect, and SciVal, RELX’s TotalPatent, and further draws on public sources, including dblp, arXiv, Stanford AI Index, kamishima.net, and Kaggle, as well as datasets provided by the Institute of Automation, Chinese Academy of Science.

Alice Bonasio is a VR and Digital Transformation Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio on Twitter.

The post Report: Global AI Research appeared first on Tech Trends.

The latest forecast from technology analyst firm CCS Insight highlights some of the challenges facing makers of head-worn virtual and augmented reality devices. Despite early enthusiasm and a continued expectation of significant long-term potential for the technology, there is much work yet to be done on improving user experience, developing software platforms and – above all – making attractive content for those platforms available to consumers.

“We continue to believe that content is the key to unlocking adoption of VR. Although some games companies and adult content creators have embraced VR technology, much more needs to happen to persuade consumers that VR devices are a must-have item,” says Ben Wood, CCS Insight’s chief of research.

This sober yet positive message is the main conclusion of the company’s recently published global market report for VR and AR devices. In it, the research firm forecasts 8 million virtual reality (VR) and augmented reality (AR) headsets and glasses will be sold in 2018, down from 10 million in 2017, but growing to 14 million in 2019 and eventually to 52 million in 2022.

Smartphone VR headsets sales have collapsed, but standalone devices like the Oculus Go hold great promise
Share on X

The market for VR has gone through several phases, according to Wood. “The first wave of consumers used basic cardboard viewers, which helped them understand the potential of the media, but the limited experience meant most people quickly lost interest. We don’t even include such devices in our forecast anymore”. He continues, “Attention then turned to smartphone-based devices such as the Samsung Gear VR and the Zeiss One Plus VR. These have played an important role in raising the awareness of the technology over the past three years, but interest in these devices has now waned”.

Accordingly, the firm’s forecast shows a dramatic drop in sales of smartphone VR devices, from nearly 8 million units in 2017 to fewer than 3 million this year. This decline has occurred despite heavy price cuts and extensive promotions over the past 12 months.

“We believe the device market has now shifted away from the early wave of cheaper, less-functional cardboard and plastic smartphone VR viewers. We expect the next wave of adoption to come from dedicated devices that offer a more compelling, wire-free experience,” Wood concludes.

CCS Insight believes the advent of standalone VR headsets – which overcome the awkwardness of being attached to a PC, smartphone or games console – will help to reinvigorate this product category both for consumers and businesses. Its forecast shows demand in that category growing over 16 times between 2018 and 2022, when 29 million standalone VR headsets are expected to be sold.

CCS Insight expects the market will return to growth in 2019, reaching 14 million devices
Share on X

The company believes devices such as Facebook’s Oculus Go and Oculus Quest, which are being offered at retail prices of $200 and $400 respectively, are good examples of the future of VR. Wood also notes that the Chinese market is looking particularly promising and Facebook’s partnership with Xiaomi has helped those headsets make a strong start there.

There is still, however, a role to be played in this market by tethered devices like the Sony PlayStation VR, HTC Vive or Oculus Rift, as currently they still offer a higher quality experience than standalone devices are able to deliver. Accordingly, CCS Insight forecasts that 5 million tethered VR headsets will be sold in 2019, growing to 18 million in 2022.

52 million VR and AR devices will be sold in 2022 with a market value of $8.5 billion
Share on X

Sony, which CCS Insight considers as the most successful manufacturer of tethered VR headsets to date, looks set to continue with its strategy of offering its VR headset as an accessory to the large number of PlayStation owners as these devices are best suited to meet the needs of the most dedicated gamers, who want a best-in-class experience.

The report also assesses the market for AR smart glasses, which have so far seen significantly slower adoption compared to VR. The main use for AR devices continues to be in business operations such as customer service, logistics, remote servicing, design and other related applications, and interest remains high in those categories. Although businesses have taken a cautious approach to AR devices, initially focusing on pilot deployments, but CCS Insight believes there are signals of growth as the chart below demonstrates.

The company believes devices such as Facebook's Oculus Go and Oculus Quest, which are being offered at retail prices of $200 and $400 respectively, are good examples of the future of VR
Share on X

“We are seeing a rising number of companies committing to deployments of tens or hundreds of augmented reality glasses as they start to see the clear benefits of head-worn technology in the workplace,” explains Wood, although he cautions that this market growth will take time, with cumulative global sales not expected to top 1 million units before 2021.

When thinking about AR glasses for consumers, Wood notes that these devices continue to be “more common in science fiction than in real life” with little evidence of a product with mass-market appeal any time soon. Nevertheless, CCS Insight’s long-range forecast does include a scenario of accelerated adoption if a major consumer brand like Apple decides to enter the market.

Yet in Tech Trends’ humble view, with Magic Leap finally entering the arena this year and a new version of the HoloLens expected to be unveiled by Microsoft next year, it could be that this tipping point might arrive even sooner, and perhaps even driven (perish the thought!) by players other than Apple. Watch this space…

We are seeing a rising number of companies committing to deployments of tens or hundreds of augmented reality glasses as they start to see the clear benefits of head-worn technology in the workplace
Share on X

For companies looking to get into Immersive technologies our Consultancy service offers support in strategic leveraging VR/AR/MR/XR

Alice Bonasio is a VR Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio on Twitter.

The post Report: VR Device Market Outlook appeared first on Tech Trends.

Even in an Antarctic research ship, ninja tech skills can come in handy, as one software engineer found in the trip of a lifetime.

A research trip around the Antarctic gave a Software Engineer Carles Pina i Estany a new perspective on how the very product he helped build was being used by scientists. Aboard a research vessel thousands of miles from shore and the nearest reliable internet connection, these modern explorers routinely relied on Mendeley to do their work, even in subzero temperatures.

A research trip around the Antarctic gave a Software Engineer Carles Pina i Estany a new perspective on how the very product he helped build was being used by scientists
Share on X

The opportunity to take part in this unique expedition came when Carles’ partner, Jen Thomas, was invited to become Data Manager on a a research trip led by the newly created Swiss Polar Institute, which has a mission to connect researchers active in polar or extreme environments, promotes public awareness of these environments, and facilitates access to research facilities in those extreme environments. As they were short of an IT person and Carles was already on a sabbatical from his work at Elsevier – during which he had already planned to travel around the world – he seized the opportunity to fulfill a longstanding ambition to take part in a polar expedition.

Aboard a research vessel thousands of miles from shore, modern explorers rely on technology to do their work in subzero temperatures
Share on X

Carles joined the Mendeley team as a developer in June 2009, back when the start-up consisted of about a dozen people working in a small office near Farringdon, a historic section of London. Four years later, the company he helped build was acquired by Elsevier. Carles, along with the founders and the now much larger team, stayed on through the transition. Now he’s a Senior Software engineer.

I first met Carles when I was also part of the Mendeley team back in 2013-15 and saw how resourceful and creative he and his colleagues were with technology (specially during the monthly hack days he helped organise). I also knew he had a penchant for adventure, as many a fascinating tale was told by the espresso machine at our old White Bear Yard office when he returned from trips to places like China and Africa.

Carles joined the Mendeley team as a developer in June 2009, back when the start-up consisted of about a dozen people working in a small office near Farringdon
Share on X

So I wasn’t exactly surprised when I learned that he had taken this rather adventurous sabbatical as IT Systems Engineer aboard the Akademik Tryoshnikov research vessel on an ambitious Antarctic circumnavigation via Cape Town, South Africa, Hobart, Tasmania and Punta Arenas, Chile.

When he returned to London and his work at Mendeley — now in the Alphabeta building in London’s Tech City — I caught up with him to find out more about the trip. As it turns out, the experience had been an eye-opener about how important the work he had done for all those years has helped to advance science. Seeing the sometimes unexpected ways researchers in this Antarctic expedition used tools like Mendeley left him with a sense of renewed appreciation for how his own work has played a part in enabling scientific research.

The opportunity to take part in this unique expedition came when Carles’ partner, Jen Thomas, was invited to become Data Manager on a a research trip led by the newly created Swiss Polar Institute, which has a mission to connect researchers active in polar or extreme environments, promotes public awareness of these environments, and facilitates access to research facilities in those extreme environments. As they were short of an IT person and Carles was already on a sabbatical from his work at Elsevier – during which he had already planned to travel around the world – he seized the opportunity to fulfill a longstanding ambition to take part in a polar expedition.

Carles joined the Mendeley team as a developer in June 2009, back when the start-up consisted of about a dozen people working in a small office near Farringdon, a historic section of London. Four years later, the company he helped build was acquired by Elsevier. Carles, along with the founders and the now much larger team, stayed on through the transition. Now he’s a Senior Software engineer.

I first met Carles when I was also part of the Mendeley team back in 2013-15 and saw how resourceful and creative he and his colleagues were with technology (specially during the monthly hack days he helped organise). I also knew he had a penchant for adventure, as many a fascinating tale was told by the espresso machine at our old White Bear Yard office when he returned from trips to places like China and Africa.

So I wasn’t exactly surprised when I learned that he had taken this rather adventurous sabbatical as IT Systems Engineer aboard the Akademik Tryoshnikov research vessel on an ambitious Antarctic circumnavigation via Cape Town, South Africa, Hobart, Tasmania and Punta Arenas, Chile.

When he returned to London and his work at Mendeley — now in the Alphabeta building in London’s Tech City — I caught up with him to find out more about the trip. As it turns out, the experience had been an eye-opener about how important the work he had done for all those years has helped to advance science. Seeing the sometimes unexpected ways researchers in this Antarctic expedition used tools like Mendeley left him with a sense of renewed appreciation for how his own work has played a part in enabling scientific research.

Here, Carles talks about those experiences and shares the photos he and Jen took during the voyage. (He explained that they were not allowed to go on deck or outside when the weather was rough, so all of the pictures are in calm weather.)

What was this expedition about, and what role did you actually have in it?

I was the software engineer on board the ACE expedition, which had 22 cross-disciplinary teams of scientists, about 150 researchers in all, who changed around at various stages of the expedition, each of which lasted about a month. Some were PhD students, some veterans of many expeditions with many years of experience, so it was a real mix.

Carles Pina i Estany diagnoses a broken hard disk. (Photo by Jen Thomas)

On the ship, each team set their own equipment up for their labs. Quite often these pieces of equipment are either connected to a computer or have a computer built into them already, to collect data and operate a machine, for example. I’d help them with things such as backing up data and any other issues that came up with those computers. I learnt a lot about oceanography and had to think on my feet because of the limited resources. Think of programming almost off-line for four months! No Google, Stack Overflow or any documentation besides the off-line versions.

Were many of those scientists using Mendeley, and how did you find them?

I knew that many of them would be Mendeley users. I do like talking to users and finding out how they use Mendeley (specially Mendeley Desktop, since that is the part of it I’ve been most involved in for many years). As they explain to me their favourite features and tell me what they would add, I can often point out useful features in the product that they are missing out on. Sometimes the features they ask for are already there, so I love helping them discover these.

 

Personally though, I like meeting users because what I do every day is for them, and it reminds me that there are humans using the software and that they have feelings. So if the software works well they love it, but they can also hate it if it ends up causing them stress, makes them lose work, miss a deadline for writing a paper or a thesis… At the end of the day, it’s very emotional and personal work, and it helps to keep that into perspective, even when you’re deep in the code. I like it when I can put faces to our users – I think in terms of “this user does this” rather than “just 23 percent of our users do this.”

How did the researchers react when they learned about your background?

Sometimes I’d get talking to them, and as I explained that this was my “year off” from my job working on the Mendeley Desktop team, their first reaction was that it was pretty intense for a year off. But the second one was that often they would say, “Oh, Mendeley Desktop – I love it! Ah, OK, OK, do whatever you want with this equipment!” So I ended up getting extra leeway for doing things around the ship because they liked and trusted Mendeley (and therefore me).

If I worked on some unrelated industry that they didn’t know about, they would have been more reluctant to let me change things around, I believe. They were really happy that I came from a place where I actually understood part of their work, what they actually do, etc. But I have to say that after this trip I now know waaaaaaaaaaaaay more than before!

An iceberg near Mount Siple in February. (Photo by Jen Thomas)

Were there any awkward moments?

One user was scared about the fact that Mendeley had been acquired by Elsevier, so we had a long talk about how Mendeley was publisher agnostic, and how that could still work in the Elsevier business model. Many users came up with long lists of ideas. Some of them turned out to be things that we already had tools for – deduplication, for example – but sometimes there were things that we could do, or that we hadn’t considered. I always encouraged them to tell me about these so I could either email them to the Desktop team back in London or consider them once I got back – which I have!

Did you get any surprising product insights during your time on the ship?

We were mostly working from the “expedition office,” where we had two “hot desks,” so lots of different people were coming and going to do their printing, use the intranet, send something over the Internet, etc. One of my favourite things to do was spot whether they were using Mendeley Desktop, and if they were, I’d sometimes ask if they liked it or not before confessing that I’d actually done part of that. I felt a bit like a “mystery shopper” or perhaps an undercover “Mendeley investigator.”

 

One really amazing thing for me was seeing how Mendeley Desktop was being used offline during the expedition because the Internet connection was almost non-existent (we only had a limited satellite link, but this was only for email and data, not normal use). It was good to see that users really appreciated that they could carry on accessing all their notes and information easily even when there was no Internet connection.

One of the researchers said to me (and I think he was only half joking) that usually he did not use Mendeley himself because he was a professor now, and “that’s the reason why I have assistants –they use it for me!” I thought, well, maybe that’s one of the reasons that some of our users stop using Mendeley – they become professors!

 

Some days, when I was feeling like talking to more researchers about Mendeley, I’d just put on my Mendeley T-shirt, and that would soon get some conversations started.

King penguins in South Georgia in March (Photo by Carles Pina i Estany)

My favourite day was when four of us were having dinner round a table – myself, my partner Jen, who was the expedition’s Data Manager, one researcher who already knew I worked at Mendeley, and one who didn’t.

The one that didn’t know where I was from saw the T-shirt and started saying, “OH WOW! MENDELEY! I LOVE MENDELEY! I use it every day! It’s fantastic! I added all my papers, I can find them, cite them, share. I really like how it’s done and what it can do! It’s AWESOME!”

So I was sitting there blushing, until she asked me where I got the t-shirt, and when I told her I worked at Mendeley, she actually jumped up and went “WHAAAAAAAAAAT??!!! YOU WORK AT MENDELEY?!?! It’s best thing ever!!!!”

As I said, it can be an emotional thing for researchers, and it sometimes feels good to be reminded of that.”

This article was originally published on Elsevier Connect

Antarctic exploration in the digital agehttps://t.co/EAkPBDwQya (w/ @mendeley_com) pic.twitter.com/Lmz0MrMvGa

— Elsevier (@ElsevierConnect) January 16, 2018

Alice Bonasio is a VR Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio on Twitter.

The post When Polar Exploration Gets Techy appeared first on Tech Trends.

Academia.edu was founded in 2008 with the mission of making knowledge more accessible and fostering discussion amongst scholars.

10 years later the platform hosts over 12 million Open Access papers and has a global community of more than 35 million registered users.

There are a lot of different types of academic output, but there’s no real incentive to share those
Share on X

Last week we stopped by their offices in San Francisco to talk to CEO and Founder Richard Price about what’s next for them, and where they see the future of academic research and education going.

“We eventually want to see all research being made freely available,” says Price. That’s an estimated 65 million papers, so he admits there’s still a way to go. The other part of his mission is to foster meaningful discussions around those papers.

Many people don’t actually read the papers they cite, so citations are not always an accurate measure of impact
Share on X

The challenge for academic social networks such as Academia.edu and its main competitors — ResearchGate and Mendeley — has always been to foster these broader conversations outside the pre-existing boundaries of departments and institutions. Academia tends to be insular and scholars are justifiably protective of their research, which can hinder openness and collaboration. Arguably, however, the next generation of researchers, having grown up in a world of social media and pervasive sharing, will find this much more natural.

The next generation of researchers growing up with social media might find sharing much more natural
Share on X

One of the features they are piloting to encourage these exchanges is called Sessions, where users can upload a draft version of their paper and invite others to discuss it within a closed group for a period of 20 days. There are about 6000 sessions happening at any one time within the site, some with as many as 1000 members (although the average group size is around 50).

“The other tool we’ve developed allows a subset of authoritative users to review papers and recommend them as valid contributions to their field of research. “Currently this is limited to a very small group of users (less than .1%), who must have a PhD and have been published, plus have experience of the peer-review process, but we are looking at expanding that pool.”

The measure of importance of an academic paper is usually tied to the number of times it gets cited, the so-called “Impact Factor”, yet price believes that platforms such as Academia.edu will allow for more granular assessment of the real-world impact that research has. It’s a complex matter, because even citations can fail to give you an accurate picture of the credibility of a particular paper, as many people don’t actually read the papers they cite.

“We need to get a lot better at asking — and answering — the question of what do academics actually think about this paper,” says Price. “I’m a huge believer in Altmetrics, and think the new measures of impact such as Author Rank and Paper Rank will become a lot more important in years to come.”

Part of that means moving beyond the PDF as the standard unit of scholarly output. There are a lot of different types of files that get produced, such as data-sets, code, blog posts, etc. Yet there is no real incentive for sharing these As we move beyond the impact factor we will get a lot more granularity and that will help paint a bigger and more accurate picture of what knowledge is relevant and credible, and to better connect people with the knowledge they need.

We don’t know how big the market is, because academic papers haven’t been openly accessible before
Share on X

The company so far raised around $17 Million and is working on ways to monetize some of its features while keeping the core platform and its content free. Yet the size of its potential user-base is actually hard to estimate, since its appeal extends beyond the traditional scholarly community.

“Only about half of our users are academics per se. The rest are teachers, diplomats, accountants, lawyers… We have investment bankers and African farmers coming to the site for the most diverse reasons, it’s really for anybody with an interest in academic research, and we don’t know how big that market is, because papers haven’t been Open Access on this scale before.”

Some of the specific use cases he mentions include a teacher using papers to support an argument for adopting a different methodology in the classroom, or a farmer looking at increasing their yield by learning about new variety of maize. And he shows me a quote that illustrates what impact this access to knowledge can have in people’s lives:

“May Allah bless you as long as the East Wind blows and shakes the branches in the distant meadows,” reads the lyric message from a grateful user in Singapore. “What you have created in academia.edu is arguably the best tool for scholars and academics. I pray to God that he provides you with all the wealth and knowledge necessary for you to keep doing what you are doing. A thousand thanks and gratitude.”

In a few years, Price hopes that all new papers published will be open and freely available so that anyone with an Internet connection can read them.

“When I was at school I only used the occasional basic AltaVista search, but in future I see a world where a kid in China interested in lithium ion batteries can find the latest research published on that subject, read it on her mobile translated automatically into Mandarin, and be able to see which of those papers are ranked as most relevant by people respected in that field, so that she instantly knows what’s most relevant to her. That’s what I want Academia.edu to make possible.”

In a few years, Price hopes that all new papers published will be open and freely available so that anyone with an Internet connection can read them
Share on X

Alice Bonasio is a VR Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio and @techtrends_tech on Twitter.

The post Sharing Knowledge Beyond the Ivory Tower appeared first on Tech Trends.