Why is User Interface/User Experience (UI/UX) so Important? It’s All in the Neuroscience
We live in a time of amazing technological advances. From the “supercomputers” we carry in our hands, and the plethora of apps that they offer, to smart appliances and automobiles, and immersive technologies such as augmented and virtual reality. Technology is advancing at an exponential rate and has the potential to increase our efficiency and quality of life. Unfortunately, too often the primary focus is on the technology, with much less focus on optimizing the interface and the experience for the user (the UI/UX). Although computing power, specialized graphics, controllers, and the like are important, their value is diminished if they don’t engage the user in a way that is natural, and in a way that makes achieving the desired goal easy.
Too often the technological “wow” factor dominates, when what is more important is how the user’s brain and body are engaged in the interest of achieving some goal Click To TweetUnnatural UI/UX increases the likelihood of failure, and results in ineffective and underutilized technology. Too often the technological “wow” factor dominates, when what is more important is how the user’s brain and body are engaged in the interest of achieving some goal. Technology for the sake of technology is not an advance, but technology that increases our efficiency by embracing the importance of optimizing the interface with the user and the naturalness of the user’s experience, offers a huge advance. To build technologies that meet these demands one must understand how the human naturally experiences and interacts with the world.
How Humans Interact with Their World
The human body and the brain that drives it, have evolved in a world rich in context. We experience and interpret the world through our senses (e.g., sight, sound, touch, smell) and interact with and affect our world through verbal and non-verbal behavior. We build devices to help us solve problems and to perform more efficiently but all too often these devices offer context-poor experiences, and require interactions that are unnatural. This increases the likelihood of failure and decreases effectiveness. For example:
- We experience and interact with a world that is 3-dimensional, dynamic, and whose views change when we shift fixation. 2-dimensional, static worlds are unnatural and less effective.
- We experience and interact with a world that is multi-sensory (sight, sound, touch, smell) and immersive. Visually dominant, small planar worlds are unnatural and less effective.
- We experience and interact with a world where we can shift our attention seamlessly, but we always have significant awareness of our surroundings. Focused attention with practical oblivion of our surroundings is unnatural and less effective.
- We experience and interact with our world and the world interacts back through sights, sounds, touches, smells, temperatures. Thumb- or finger-only interactions are unnatural and less effective.
We must build devices that offer a context-rich user interface, and natural user experiences. To achieve this aim, we need to understand how the human being and its brain have evolved to experience and interact with the world, and to emulate that as much as possible when developing software and technology. One must remember that UI/UX is about the user interface and the user interaction. It is about the user. To understand the user, one must understand the user’s brain; how the brain processes information to achieve goals, and how the brain interacts with technology to achieve goals.
The Psychology and Neuroscience of Human Experience and Interaction
The human brain is comprised of at least four distinct systems that define how humans experience and interact with the world. The experiential system has evolved to represent the sensory and perceptual aspects (whether visual, auditory, tactile, or olfactory) of an experience or interaction with the world. These are the sights, sounds, touch and smells in our environment. The brain processes sensory information in distinct brain regions, but critically the perceptual representations are highly interactive, and yield a rich contextual framework for each interaction. The critical brain regions associated with the experiential system are the occipital lobes (sight), temporal lobes (sound), and parietal lobes (touch and smell).
One must remember that UI/UX is about the user interface and the user interaction. It is about the user Click To TweetThe cognitive system is the information processing system. It takes sensory and perceptual information, usually in the form of text or graphics, and translates that into information that can then be acted upon. The cognitive system relies heavily on working memory and attention that are both limited capacity and form a processing “bottleneck”. One must always be wary not to overload this system as overloading will lead to errors and inefficiency. This system encompasses the prefrontal cortex and hippocampus.
Neuroscience of Human Experience and Interaction
The behavioral system in the brain has evolved to drive motor behavior. These include moving one’s arms, hands, or mouth in such a way to lift, move, type or speak. This system learns behaviors when they are followed in real-time (literally within milliseconds) by corrective feedback that rewards correct behaviors and punishes incorrect behaviors. Once learned, behavioral repertoires are elicited by context-rich environments that have been associated with particular behaviors. In short, this system learns stimulus (e.g., rich environmental context) – response (e.g. speech or motor movements) mappings. This is why certain contexts elicit behaviors without the user even “thinking” about it. This system relies on the striatum, and dopamine-mediated reward and punishment signals.
The emotion-motivation system in the brain has evolved to process positive and negative emotions (e.g., happiness, sadness, anger, frustration, regret, etc.) and motivations such as rewards and punishments Click To TweetThe emotion-motivation system in the brain has evolved to process positive and negative emotions (e.g., happiness, sadness, anger, frustration, regret, etc.) and motivations such as rewards and punishments, as well as stress, pressure and anxiety. Positive emotions and motivations can enhance cognitive processing and negative emotions and motivations can dampen cognitive processing. Positive and negative emotions and motivations define the rewards and punishments that drive behavioral learning. The critical brain regions are the amygdala and other limbic structures.
Optimized UI/UX:
Narrowing the Gap Between Human Experience and Interaction with the World, and Human Experience and Interaction with Technology
One thing that is clear from the psychology and neuroscience of human experience and interaction is that it is multi-faceted. It is rich in sensory inputs and perceptual representation. It is 3-dimensional, dynamic, and immersive. Although cognitive and information processing is important, it is only one part of human experience and interaction, and it is also the most fragile because working memory and attention are limited capacity resources. One should not expect this system to do the heavy lifting without significant support. Humans are highly interactive social and behavioral machines. We interact and communicate effectively through speech and non-verbals. We also have a strong behavioral repertoire from gross motor movements like lifting, to fine motor movements like typing or threading a needle. Because our behavioral repertoire is so broad, we should leverage this as we build the UI/UX for technologies. Finally, we are emotional organisms that use emotion and motivation to drive behavior, to remain engaged, to relate to others and to persevere through hard times.
Because our behavioral repertoire is so broad, we should leverage this as we build the UI/UX for technologies Click To TweetTo optimize UI/UX with technology we need to keep the psychology and neuroscience of human experience and interaction at the forefront. Every step that we take that narrows the gap between how humans experience and interact with their world and how they experience and interact with technology, is a step in the right direction. In general, technologies with the best UI/UX are those that broadly and naturally engage multiple systems in the brain in synchrony, and in a way that minimizes cognitive load, enhances engagement and motivation, all while optimizing the speed and accuracy of motor behavior.
Experiential system: Every interaction with technology is an experience. You do not want UI/UX to focus exclusively on 2-d, static, visual displays, with thumb presses, or swipes as the only form of interaction from the user, and new 2-d, static visual input as the only response from the technology. You want experiences to be rich in context. The more 3-dimensional, dynamic, and multi-sensory the better. Users should interact with technology using visual fixation (without touch), speech, and motor behaviors that use more than the thumbs. Technology should respond to users with a rich array of color, form, motion, sound, and tactile assets. These should be constructed in such a way as to optimize the user’s experience and interaction. The user needs to interact with the technology, but the technology needs to interact with the user.
You want experiences to be rich in context. The more 3-dimensional, dynamic, and multi-sensory the better. Users should interact with technology using visual fixation, speech, and motor behaviors that use more than the thumbs Click To TweetThe more sensory and perceptual systems are engaged in the interest of achieving the desired goal, not simply to increase the “wow” factor, the more memorable and enjoyable the experience, and the more likely the user is to achieve the goal quickly and accurately. Technologies like augmented and virtual reality hold great promise for engaging the experiential system. Augmented reality does this by overlaying assets onto the real world. Virtual reality does this by immersing the user in a new, virtual world. Technologies that utilize speech recognition also address these issues (e.g., Alexa, Siri, Google). Technologies that utilize wearable sensors or some other method of tracking full hand and arm movements as part of the user interface will be more natural for users. In addition, the plethora of sensors that track the user’s state of readiness including fatigue, stress, and vigilance, can be utilized to more effectively interact with and guide users.
Technologies like augmented and virtual reality hold great promise for engaging the experiential system Click To TweetCognitive system: Too much UI/UX taxes this system without providing support from other systems. Too often technology is purely visual, and requires so much attention and working memory from the user, that they are functionally oblivious of their surroundings. This is hazardous, but also means that any lapse in attention or overload of working memory will lead to failure. The goal is to reduce the cognitive load by providing support from other systems and by providing users with what they need, where they need it, and when they need it. Again, technologies like augmented and virtual reality will be useful here, as their assets can be overlaid on the real or virtual world in a way that minimizes cognitive load and provides an experiential scaffolding.
Even so, much more attention is needed to reduce cognitive load, even with traditional 2d, planer technologies. Breaking a task down into small, manageable units (much like microlearning in Learning & Development) can reduce the cognitive load, and increase the chance of success. Every incremental increase in cognitive load is an incremental increase in difficulty, likelihood of failure, and likelihood of abandoning the technology. Technologies that are effective at achieving these goals are usually the most successful. Although perfection has not been achieved, and advances are still necessary, incorporating speech, natural language processing, audio and more natural behavioral interfaces will go a long way toward addressing some of the bottleneck associated with cognitive load.
Advances are still necessary, incorporating speech, natural language processing, audio and more natural behavioral interfaces will go a long way toward addressing some of the bottleneck associated with cognitive load Click To TweetBehavioral system: Technologies that optimize the naturalness of behavioral interaction are rare. This is surprising given the large body of groundbreaking research on procedural and motor skills. With the majority of UI/UX being haptic and click- or touch-based, understanding this system is especially crucial. Three critical findings are relevant. First, a high-quality user interface builds a collection of relevant behavioral repertoires through real-time immediate feedback that will facilitate interaction with the technology today and will enhance the ease of interaction in the future. This means that appropriate behaviors should be clearly rewarded, and inappropriate behaviors should be clearly punished, in real-time. Simply moving on to the next step following a correct action is not enough. Provide a highly motivating, immediate feedback signal, and that behavior will be more likely to follow in the same context next time. Second, a high-quality user interface implements related tasks with similar motor behaviors and unrelated tasks with dissimilar motor behaviors. Motor skills can be complementary or competitive and this should be incorporated directly into the UI/UX.
Motor skills can be complementary or competitive and this should be incorporated directly into the UI/UX. Click To TweetIf two interactions with a technology achieve a similar goal, then the motor behavior to achieve those goals should be similar (e.g., to accept some choice, swipe right). On the other hand, if two interactions achieve different goals, then the motor behavior should be qualitatively different (e.g., swipe right to accept and swipe left to reject). Finally, a high-quality user interface needs to use natural motor behaviors to achieve goals. If my goal is to use virtual reality to practice a medical procedure that does not involve button presses, then there should be no button presses as part of the user interface. Too many virtual reality training tools show realistic virtual objects, but the interaction is not realist and is unnatural. This trains associations between unnatural behaviors and outcomes. We need to move beyond thumb- and click-based behaviors, and beyond unnatural hand controllers. Speech and full hand gestures must be incorporated using sensors and other methods of tracking.
Emotion-motivation system: UI/UX that effectively engages the emotion-motivation system is critical. When training behavior the UI/UX must be such that behaviors are rewarded or punished in real-time. Without this behavioral learning will be suboptimal. Technologies like virtual reality are especially effective at engaging emotion and motivation centers. The ability to be transported into any situation, whether emotion-laden or not, is critical for building situational awareness that is so important in so many aspects of our lives. In addition, the ability to be transported into another person’s shoes, to see the world from multiple perspective, and to “walk a mile in someone else’s shoes” will go a long way toward building the people (aka soft) skills and empathy that are central to success and to a positive workplace and society.
UI/UX that effectively engages the emotion-motivation system is critical. When training behavior the UI/UX must be such that behaviors are rewarded or punished in real-time Click To TweetThe immersive technologies of virtual and augmented reality go a long way toward addressing many of the UI/UX concerns raised above. First, and foremost, they are grounded in context-rich experiential learning that is 3-d, immersive and dynamic. This is our natural state. Immersive technologies offer the promise of minimizing cognitive load, but often there is a conflict between efficiency due to reduced cognitive load, and “flash”. Good solid iterative experimentation can address this concern. Although there is still work to do to incorporate truly natural haptics, as well as speech recognition, and real-time measures of physiology, augmented and virtual reality offer great promise as tools to train behavior. Finally, immersive technologies, especially virtual reality, can train situational awareness, people skills and empathy by effectively engaging emotion centers in the brain.
The most important take-home message, and one that continues to be missed by so many, is that technology is there for the user. The best technology conforms to the user’s natural tendencies and leverages those to achieve goals. The worst technology requires the user to conform, and expects unnatural tendencies to get the job done. The more we place the user at the center, and not the technology, the better.
