Microsoft’s new piece of kit doesn’t disappoint, but the real excitement of its MWC announcement goes well beyond the hardware.

After unveiling the device at MWC, Microsoft then went on to demo it to a select group of invited media (myself included) and that’s where I finally got the chance to play with what has to be one of the world’s most sophisticated toys, certainly for that price range.

Microsoft is in this for the long haul and will continue cashing in just fine without even touching the consumer market
Share on X

I won’t go over the basic specs here. After tons of excited reporting, the device itself – and Microsoft’s strategy for it – have been thoroughly analyzed from every angle. From in-depth technical reviews to accounts of testing the prototype in Redmond prior to the launch, all the way to the nay-sayers who still insist that the whole thing is doomed to fail, take your pick. What I’m interested in, however, is what this means for the broader immersive tech ecosystem.

The HoloLens and I go way back. Ever since I demoed the device in London for the first time back in 2016 I was convinced this was going to be something truly transformational. I went on to report on many different use cases that demonstrated just how much value the Mixed Reality proposition could bring to all sorts of industries.

The HoloLens and I go way back
Share on X

While others griped endlessly about the field of view and somewhat stilted gestural interface, I always preferred to focus on the bigger picture. The HoloLens was essentially an excellent proof of concept, and much of its technology was still amazingly ahead of its time even four years after launching. I had the opportunity to deep-dive into how the Microsoft team painstakingly developed the awesome spatial audio feature to heighten immersion, and interview its creator Alex Kipman on his vision for the future of both the device and the tech.

So after hearing all about it at the launch in Barcelona, I was keen to see it with my own eyes. Would it actually deliver on the promises Kipman had made on stage the night before?

While others griped endlessly about the HoloLens field of view and somewhat stilted gestural interface, I'd rather focus on the big picture
Share on X

When I first put the headset on, we were off to a good start: Promise number was kept right off the bat. This was vastly more comfortable (I’m not sure if it was exactly “three times more comfortable” like Kipman claimed but it was, well, comfortable!). Whereas I usually came away from HoloLens demos with big red marks on the bridge of my nose, the weight of this new device was elegantly distributed and the center of gravity much further towards the back. Combined with lighter carbon fiber construction, this is something I could happily wear for a while without thinking anything of it.

The second big promise Kipman had made was that this device was much more immersive. The fact the Field of View (FOV) is twice the size of the old HoloLens certainly helps towards that. Although I maintain that even with the original FOV your brain adjusted pretty quickly to it so that you still got an immersive experience, it was nice not to have to adjust. The FOV on the HoloLens 2 does not quite cover your entire line of sight, but it really does come close enough so that it doesn’t interfere with the experience. I spoke to Mark Christian, Global Director of Immersive Learning at Pearson – one of Microsoft’s HoloLens Partners – who told me that after delivering over 200 demos on the first day at MWC (conference attendees were waiting over 3 hours in line for the chance to try the device) not one of them mentioned feeling limited by the FOV.

So now that we got that out of the way, here’s what got me most excited about my own demos: The first couple of minutes in each one, where you’re essentially calibrating the device.

The second big promise Alex Kipman had made was that the HoloLens 2 was much more immersive
Share on X

By the third and fourth time I was doing that, I knew what to expect, but it was still magic, and to me it really sums up why this is an awesome leap forward for Microsoft and for Mixed Reality. It works very nicely across the board, but where it comes to eye tracking and voice recognition (and the combination of the two) it delivers in spades. And then some.

The first calibration exercise (after you do the familiar lining up of a holographic box onto the middle of your display so you can see all four edges) has a series of bright spinning jewels pop up in different places in your field of view. I’m instructed to just look at them, without moving my head. As my eyes rest briefly on each jewel, it disappears in turn. The HoloLens 2 knows exactly where I’m looking. After that’s done, a colorful hummingbird materializes in front of me. The device scans my hands (automatically adjusting to their shape and size) and when I move them the tiny bird flies over to hover above it. I change hands and move them around, feeling like Snow White in a Disney film as I make friends with this little guy, who I really feel like naming. By the time we finish calibrating the device, I regretfully watch him disappear.

But the idea behind having me play with this hummingbird is, of course, for the HoloLens to be able to scan your hands and their movement patterns to allow you to interact with the holograms. And that interaction happens in a much more intuitive flow now. Gone are the “pinch” and “bloom” gestures you had to learn and instead you can pretty much grab, stretch, toss and turn your holograms any way you please. You can push buttons, slide sliders, and tell stuff to do things. And because the device always knows where I’m looking, I can often do what I want just by directing my gaze somewhere. The prime use case of that is incredibly simple, and beautiful in its simplicity – reading. Text information was set next to a Hologram, and as my eyes finished reading the last line, that text automatically scrolled up. Not only that, but it did so faster or more slowly depending on my reading speed. It was absolutely seamless, and one example of what Microsoft CEO Satya Nadella referred to in his speech on Sunday at MWC, that the best technology eventually becomes invisible.

The idea behind having me play with this holographic hummingbird was to have the HoLens 2 scan my hands
Share on X

And as for value, the price of the HoloLens 2 is actually incredibly cheap for most the large enterprises that have been using it so far. The cost and efficiency savings that introducing Mixed Reality have already brought to companies like Thyssenkrupp and Trimble are nothing short of remarkable, and the possibilities for increasing those returns even further are practically endless. That’s the low-hanging fruit as far as Microsoft is concerned. I have a feeling the only question those companies will be asking themselves is “how many of these will we be allowed to order, and how quickly can you deliver?”

But Microsoft also wants to create this “open ecosystem” around Mixed Reality and expand its appeal to small and medium-sized businesses. And for SMEs, the fact that the HoloLens 2 will also be offered on a lease basis for a few hundred dollars a month is a big attraction. The key, however, is going to be the out-of-the-box content that they will be able to offer those companies. And this is again where Microsoft’s strategy really comes together. For the past four years they have worked with these big corporations and their R&D departments to create valuable use cases, platforms and applications for Mixed Reality. If they’re now able to package that as part of a subscription offering for SMEs it could prompt a lot of smaller business owners to try it out.

Microsoft wants to create an open ecosystem around Mixed Reality
Share on X

So from a business perspective, there were very few surprises to come out of these announcements and the launch at MWC. Microsoft has proven itself to be steadfast and consistent in its strategy for the HoloLens and Mixed Reality, sticking to its original pivot towards enterprise.

But at the same time, there is no doubt that this technology will not be pigeonholed in such a way forever. It will evolve, become lighter, cheaper, and ever more intuitive and transparent. And when the time, price, and most importantly, the content ecosystem is right, it will reach that sweet spot where consumers will also embrace it. The HoloLens 2 brings us a big step closer to that, but there’s no rush as far as Microsoft is concerned. They’re in it for the long haul, and in the meantime I have a feeling they will start cashing in just fine without even touching the consumer market, thank you very much.

This article was originally published on VRScout

Hands-On With The HoloLens 2 via @alicebonasio https://t.co/GuvVS1iTvG pic.twitter.com/xpqRWxo6TF

— VRScout (@VRScout) March 3, 2019

Tech Trends’  Virtual Reality Consultancy services offers support for companies looking to enhance brand strategy with immersive technologies such as Virtual, Augmented and Mixed Reality

Alice Bonasio is a VR and Digital Transformation Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. You can follow @alicebonasio on Twitter, connect with her on LinkedIn or subscribe to her Inside VR/AR Newletter for all the latest curated immersive news.  

The post Getting Hands-On with the New HoloLens appeared first on Tech Trends.

Object Theory are pretty much the oldest Mixed Reality company around, having launched in June 2015, before the HoloLens had even started shipping. In many ways they’re your typical tech company – my demos are delivered in the familiar blueprint of a trendy open-plan office with exposed brick walls, which sits above a sushi restaurant – but it’s got a different vibe from a lot of similar start-ups.

It was really interesting to see Microsoft pivot towards enterprise with the HoloLens and I think it was absolutely the right move for them to make
Share on X

For starters, they’re based in Oregon as opposed to Silicon Valley. And while Portland is by no means a backwater, it’s still not a major tech or immersive content hub like LA or San Francisco either. For another, its founders Raven Zachary and Michael Hoffman (who left Microsoft to start the company) operate a very profitable business (employing about a dozen people between full timers and contractors) with no investment capital. In spite of the fact we’re joking around and playing Angry Birds on Magic Leap, the whole thing feels very grown up, in the sense that these guys are in this for the long haul, and so, they reckon, is the business of making Mixed Reality.

You haven’t really experienced how much fun @AngryBirds is until you tried it in #mixedreality. Thanks @ObjectTheory for letting me play with your @magicleap! pic.twitter.com/UMhQbxgH1g

— Alice Bonasio (@alicebonasio) February 2, 2019

A key enabler for Object Theory’s success has been Microsoft’s strategy for marketing, supporting, and developing Mixed Reality content for the HoloLens, which is why Zachary and Hoffman are enthusiastic in their praise for the company, and in particular of the leadership of CEO Satya Nadella. By pivoting towards the enterprise market early on, Microsoft managed to create strong and sustained demand for Mixed Reality tools among companies looking to solve real business needs. That, in turn, allowed their partner developers to secure key contracts as they figured their way around the new technology.

“It’s curious how the HoloLens originated with the Xbox team (its inventor Alex Kipman was also responsible for the Kinect) so there was this idea that it would be mainly a consumer product,” recalls Hoffman. “It was really interesting to see them pivot in that way and go mainstream towards enterprise and I think it was absolutely the right move for them to make.”

The other significant pivot Microsoft made in recent years, adds Zachary, was to move away from being hardware-centered to focusing on the cloud, marketing Azure’s ability to enable what they call ‘The Intelligent Edge’: “Microsoft is the only one of the large players that has actively decided to be a multi-platform company. They are actively embracing everything that’s relevant out there, and that makes sense, because they’re making cloud consumption more valuable if it works with everything that’s out there. Because of that we hope – and it would make sense – if they adopt an OEM for their Mixed Reality technology. Microsoft has this great patent portfolio and it would be great to take that amazing secret sauce of the HoloLens and license it out to their existing OEM partners like Dell or Samsung.”

Object Theory are the oldest Mixed Reality company around, having launched in June 2015, before the HoloLens had even started shipping
Share on X

Since the launch of Magic Leap One earlier this year, Object Theory has also started exploring the possibilities that the other platform brings, such as better eye tracking and support for finer and more nuanced gestural controls as well as the much-talked-about additional field of view:

“People tend to focus on the extra field of view, but part of it is just an illusion, because they just don’t let you see what is not there, it’s really a trick because the edges of your vision are restricted by the design of the Magic Leap One HMD,” explains Hoffman. “With that restriction, you never get that cognitive dissonance of having an open FOV and a restrained one at the same time, which is what you get with the HoloLens. But that said, everyone says that after a day or so of using the HoloLens your brain adapts to it and you just don’t notice it any more, so for me this is really not a major issue.”

They muse that Magic Leap probably waited too long to launch, so that by the time they did, people were both less awed by the technology, and had unrealistic expectations fuelled by the company’s infamous cinematic concept videos.

“We in the industry knew that’s what we were going to get, but consumers felt let down because they didn’t realize the whale jumping in the school auditorium was a concept video, so for us the troll throwing a boulder was fine, but consumers – who weren’t going to buy the device anyway – felt let down,” says Zachary.

If we do get a sneak peek at the HoloLens 2.0 in Barcelona on the 24^th, it will be over three years after it was first launched, so developers like Object Theory are keen to see an acceleration of that innovation cycle. Zachary and Hoffman believe that bringing competition is one way to achieve this, and that, in a way is where the real value of having Magic Leap enter the arena lies.

People tend to focus on the extra field of view for Magic Leap, but part of it is an illusion, because they just don’t let you see what is not there, it’s restricted by the design of the HMD
Share on X

Although they haven’t deployed on Magic Leap yet, they’re hopeful that because every platform out there is investing in enabling Unity compatibility which enables them to develop cross-platform more easily. The challenge going forward, they anticipate, is that major tech companies are very used to owning their developer ecosystems outright,

“The challenge for Microsoft, Apple and Google is that they’re used to owning their developer ecosystems outright,” says Zachary, “so this is the sort of abstraction that I’m not particularly sure Apple and Google will buy into – but Microsoft might because they care about the cloud more than control over the developer ecosystem. I think ultimately for us, when an enterprise customer asks us if we can deploy on Magic Leap – which hasn’t happened so far – we’re ready and open to having that conversation. We’ve been saying all along we want more players, more competition. The more success we have in making this solve real challenges or even desires such as entertainment, the more it becomes meaningful, the pie gets bigger and there’s more opportunities for companies like us.”

The main question they’re asking themselves now is whether CIOs at Fortune 100 companies will want to invest in a start-up’s hardware solution or whether they’re by default going to go with a major player such as Apple or Microsoft.

“I don’t know the answer to that, we’re still leading with the HoloLens, it’s still our platform of choice, but we don’t know where that market is, Zachary says, but Hoffman goes a bit further, outlining how Microsoft’s targeting of enterprise proved to be a genius move in rallying developers like them to the platform in spite of its relatively small install base.”

The main question they’re asking themselves now is whether CIOs at Fortune 100 companies will want to invest in a start-up’s hardware solution
Share on X

“No matter how much money the investment in Magic Leap sounds, it’s still small compared to a company like Microsoft. They are a start-up,” says Hoffman, “And if I’m an enterprise I definitely want to take the Microsoft solution because even if they’re not perfect, they’ve been solving my business needs for a very long time,” he says, adding that while Microsoft might not be perfect, they’ve become very good at providing the services their enterprise customers want, and it was unlikely those customers would look elsewhere for them. In other words, Microsoft really isn’t going anywhere, where even a very well-funded start-up like Magic Leap could eventually run out of money and disappear without a trace. Given how risk-adverse large corporations tend to be, it’s clear what an advantage Microsoft has here in continuing to develop that market.

Even after three years, a lot of the R&D behind the HoloLens still stands up well, such as the fact that the HMD allows users to wear their prescription glasses is a huge plus (Magic Leap’s design requires you to order and purchase a special insert for around $200) as is heat dissipation; “Magic Leap makes me sweat within minutes of putting it on,” Zachary says. Another feature where the HoloLens technology scores highly with the developer community is spatial audio, which is something that Microsoft invested heavily in getting just right.

“What I love about the spatial audio feature as a developer is that you literally just have to push a button and it works. The Algorithms fill in all the blanks for you,” Zachary enthuses. “The sense of presence with Prism (the Mixed Reality collaboration and productivity platform that Object Theory developed for the HoloLens) because of the spatial audio is amazing. I want to talk to the silly cartoon avatar because the sound makes it really feel like that person is in the room with me, it’s unbelievable.”

“So overall – and we asked a lot of other people about this too – the sense is that Magic Leap outperforms the HoloLens in a couple of areas, and does worse in others, but there isn’t this sense that the needle has been moved dramatically,” Hoffman adds. “We’re very curious to see how they position Mixed Reality in 2019 and to what extent HoloLens will be a core or ancillary part of that story, and whether we’re going to see OEM devices. The more partners out there building devices, the more this market is going to move forward. We were a bit early, but we’re true believers.”

This article was originally published on UploadVR

Between Magic Leap and HoloLens, which is going to win over developers?https://t.co/X6QOjv8BGe pic.twitter.com/R5EyHZQLby

— UploadVR (@UploadVR) February 20, 2019

For companies looking to get into Immersive technologies our VR Consultancy service offers comprehensive support in strategic deployment of Virtual, Augmented and Mixed Reality

Alice Bonasio is a VR Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio on Twitter.

The post Developing the Future of Mixed Reality appeared first on Tech Trends.

The Brazilian-born inventor gets recognition for his groundbreaking work with the Microsoft Hololens.

The European Patent Office (EPO) announced that it has nominated U.S.-based inventor Alex Kipman as a finalist for the European Inventor Award 2018.

“Alex Kipman’s invention is a vision for the future of computing with the potential to change how we connect and collaborate,” said EPO President Benoît Battistelli. “His ideas are enhancing the real world with digital information that helps people achieve higher levels of efficiency, and creating new opportunities in fields such as engineering, communications, and healthcare.”

Kipman started his professional career at Microsoft’s headquarters in Redmond, Washington, right after graduating from the Rochester Institute of Technology in 2001. In a career spanning more than 15 years as a developer for the company, he has been named as the primary inventor on more than 150 U.S. and Asian patents, as well as two granted European patents and more than a dozen European patent applications. His current position as Technical Fellow of the Windows and Devices Group enables him to maintain a hands-on approach to building the HoloLens and related Windows Mixed Reality software platform.

Built-in machine learning and AI-algorithms enable the HoloLens to be contextually aware of its surroundings and interpret user commands
Share on X

The HoloLens incorporates several patent-protected Microsoft inventions, including Kipman’s motion sensors initially brought to market for the Xbox Kinect video game controller. The software for the HoloLens is developed in the Windows 10 operating system in the Windows Mixed Reality software environment. Current applications of the device include telecommunications (e.g. for enhanced video calls), a 3D virtual tourism application, and a suite of mixed-reality games, as well as industrial uses for metering and quality control, and interactive digital human anatomy. A host of promising new applications are being developed. In 2017, a Spanish team of surgeons used the mixed-reality headset successfully while operating on a patient with a malignant muscular tumour. During the procedure, the surgeons were able to view live diagnostic imaging from MRI and radiography data feeds, without taking their eyes off the patient.

The HoloLens is a stand-alone device, independent from external PCs, tablets or smartphones
Share on X

Thanks to built-in machine learning and AI-algorithms, the HoloLens is contextually aware of its surroundings and able to interpret user commands such as “move that chair.” It observes its surroundings via an inertial measurement unit (a device that measures linear and angular motion), a depth-sensing camera, a 2.4-megapixel video camera, four microphones and an ambient light sensor – so it “knows” which chair the wearer is currently seeing. It collects several terabytes of data every second, piped in via sensors and then computed.

Alex Kipman's invention is a vision for the future of computing with the potential to change how we connect and collaborate
Share on X

“Current-generation mixed-reality glasses include “smart” ski goggles that project data such as altitude, temperature and pulse into the wearer’s field of vision. But these devices display only two-dimensional text and graphics, while requiring a connection to a smartphone for computing power,” explains Battistelli. “The HoloLens is a stand-alone device, independent from external PCs, tablets or smartphones. It contains all necessary computing power in its custom-built processors. These include a CPU (central processing unit), GPU (graphics processing unit) and custom HPU (holographic processing unit), an industry first.”

These processors collect several terabytes of data every second as the HoloLens reads its environment. Data arrives through an inertial measurement unit (IMU), a depth-sensing camera, a 2.4-megapixel video camera, four microphones and an ambient light sensor.

HoloLens owes its innovative edge to the waveguide lenses that trick the human brain into interpreting photons as solid, three-dimensional things
Share on X

As the name suggests, the HoloLens owes its innovative edge to the waveguide lenses that “trick” the human brain into interpreting photons as solid, three-dimensional things – hence holograms. They are computed on the GPU and displayed through the device’s “light engines”, two tiny liquid-crystal-on-silicon projectors which beam light onto the two lenses.

Processors collect several terabytes of data every second as the HoloLens reads its environment
Share on X

The image is adjusted between the two lenses for clarity and depth, then blended with the real world. As a result, holographic images appear to exist in front of the wearer – from a few centimeters to many meters away – with true depth of field.

Once ready for mass adoption, the HoloLens could lead to a fundamental shift, as mixed-reality devices replace today’s smartphones as the main communications devices. According to experts at market researcher Gartner, 30% of web browsing sessions could be done without a screen by 2020. At that point, 50% of all search queries could originate, not from keyboards or touchscreens, but voice commands as computing moves into an era beyond screens.

Once Microsoft has built an attractive mixed-reality library, products such as the three-dimensional teleconferencing app Holoportation could well join bestsellers like Office 365
Share on X

Consulting firm Digi-Capital estimated the world market for augmented- and virtual-reality devices at EUR 3.3 billion in 2016. The firm projects that, driven by next-generation mobile augmented-reality hardware, the market will reach EUR 85 billion by 2021. Once Microsoft has built an attractive mixed-reality library, products such as the three-dimensional teleconferencing app Holoportation could well join bestsellers like the Office 365 commercial suite with over 100 million users.

“Kipman’s HoloLens can make holographic likenesses appear in thin air with new levels of sophistication – for instance in teleconferencing calls. Because the mixed-reality headset requires no computer screen or keyboard, it could be part of a revolution that would replace smartphones as the main communications devices,” adds Battistelli.

Once ready for mass adoption, the HoloLens could lead to a fundamental shift, as mixed-reality devices replace today's smartphones as the main communications devices
Share on X

“It’s an experience where content is no longer tied to a screen but exists in both time and space laid over the real world,” says Kipman, who has predicted the “death” of the smartphone and a new revolution in personal computing build around the concept of Mixed Reality. Current industry trends appear to support his statement – such as the shift towards voice search with digital assistants like Cortana from Microsoft, Siri from Apple and Alexa from Amazon. Consumers may one day be talking and gesturing to holographic “ghosts”, while the smartphone becomes a spectre of the past.

Kipman has been awarded the Popular Mechanics Breakthrough Award (2009), Microsoft Outstanding Technical Achievement Award (2012), and IPO Education Foundation’s Inventor of the Year Award (2012). He also received the honor of being named one of TIME magazine’s Top 25 “Nerds of the Year” (2010), an epithet he wears proudly.

To find out how to leverage VR/AR/MR in your enterprise, Tech Trends offers bespoke Virtual Reality Consultancy support

Alice Bonasio is a VR Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio and @techtrends_tech on Twitter.

The sexily named patent is viewable in full here: EP3028121

The post Alex Kipman Nominated for Inventor’s Award appeared first on Tech Trends.

The Microsoft HoloLens inventor talks about how Mixed Reality technology will fundamentally change the way humans and machines interact.

Alex Kipman knows about hardware. Since joining Microsoft 16 years ago, he has been the primary inventor on more than 100 patents, including Xbox Kinect’s pioneering motion-sensing technology that paved the way for some of the features in his latest creation, the holographic 3D headset called the HoloLens.

The key benefit of technology is its ability to displace time and space
Share on X

But today, sitting in his office in Microsoft’s headquarters in Redmond, Washington, Kipman is not talking hardware. He’s discussing the relationship between humans and machines from a broader philosophical perspective. Whether we interact with machines through screens or stuff that sits on our heads, to him, it’s all “just a moment in time.”

---

MR has the amazing potential to unleash displacement superpowers onto the real world.
Share on X

---

The Brazilian-born Kipman, whose title is technical fellow at Microsoft’s Windows and Devices Group, enthusiastically explains that the key benefit of technology is its ability to displace time and space. He brings up “mixed reality” (MR), Microsoft’s term for tech that mixes real-world with computer-generated imagery and will, some day, according to Kipman, seamlessly blend augmented and virtual reality. He says that one of the most exciting features of MR is its potential to unleash “displacement superpowers” onto the real world.Humans attach value to the feeling you get when physically sharing a space with another person, which is the reason I took a 10-hour flight to have a face-to-face conversation with Kipman. “But if you could have this type of interaction without actually being here,” he says, “life suddenly becomes much more interesting.

“My daughter can hang out with her cousins in Brazil every weekend, and my employees don’t need to travel around the globe to get their job done,” he continues. “With the advent of artificial intelligence, we could still be talking, but I’m not even here anymore. One day you and I are going to be having this conversation, you’ll be sitting on Mars, and I’ll have been dead for 100 years. Our job as technologists is to accelerate that future and ask how we do that.”

Microsoft is betting on mixed reality to help launch us into the future. Which brings us back to hardware. The availability of the right device at the right price will be a factor in whether consumers adopt MR (though devices alone aren’t likely to jump-start a MR revolution, if the slow sales of VR systems are any indication). While the HoloLens is the only self-contained holographic computer on the market (unlike the Oculus Rift or HTC Vine, it doesn’t need to be attached by cables to an external device), the $3,000 smart glasses have served more as proof of concept than a consumer product.

---

As Microsoft sees it, introducing a platform that lets anyone in the general public build their own digital world is the first step in achieving that leap into the world of tomorrow
Share on X

---

Now Microsoft wants to change that. This fall, the company is launching the Windows Mixed Reality Headsets, its first major attempt to sell the concept to the general public. Though still closer to virtual reality than a perfect AR/VR hybrid, the new device repackages some of the main features of the HoloLens—such as its advanced tracking and mapping capabilities—at the more affordable price range of $300-$500. The headsets will be available in different forms via a number of hardware partners, including Dell, HP, and Samsung, and will enable users to create 3D spaces that they can personalize with media, apps, browser windows, and more.

---

Mixed reality devices need to support virtual imagery that seems to be a plausible part of the real world and act in a cohesive way with it
Share on X

---

As Microsoft sees it, introducing a platform that lets anyone in the general public build their own digital world is the first step in achieving that leap into the world of tomorrow. “If you believe, as we do, that mixed reality is the inevitable next secular trend of computing, it’s going to involve productivity, creativity, education, and the entire spectrum of entertainment, from casual to hardcore gaming,” Kipman says.

Perfecting Mixed Reality

Kipman is not the only one who’s bullish about mixed reality. The California-based startup Avegant is working on a platform that presents detailed 3D images by stacking multiple focal planes, which the company calls “light field” technology. “Applications are endless,” says Avegant CEO Joerg Tewes, “from designers and engineers directly manipulating 3D models with their hands, to medical professors illustrating different heart conditions through a lifelike model of the human heart for their students. At home, consumers might find themselves surrounded by virtual shelves full of their favorite products. Mixed reality enables people to interact directly with their ideas rather than screens or keyboards.”

---

In shared virtual environments, our relationship with computing changes from a personal to a collaborative one—from devices storing your own individual content, to common creative spaces mediated by technology
Share on X

---

Yet in order to do all of that, mixed reality devices need to support virtual imagery that seems to be a plausible part of the real world and act in a cohesive way with it. According to Professor Gregory Welch, a computer scientist at the University of Central Florida, most of the technology developed so far has yet to achieve that balance. “MR is particularly difficult in that respect because there is no hiding the imperfections of the virtual, nor the awesomeness of the real,” he says via email.

 

---

Virtual content is much more impactful when it behaves in the way we would expect physical objects to behave in the real world
Share on X

---

He and his research collaborators found that in some cases, the relatively wide real-world view afforded by the HoloLens could harm that all-important sense of presence. Where a healthy human can see approximately 210 degrees, the display of the HoloLens only augments the middle of your field of view 30 degrees or so. In the experiments that Welch and his team conducted, the disconnect between the real and augmented landscape diminished the sense of immersion and presence in their subjects.

“That means that if you are looking at a virtual human in front of you (as we did in our experiment), you will only see a portion of them floating in space in front of you,” Welch says. “You have to move your head up and down to ‘paint’ a perception of them, as you cannot see the entire person at once, unless you look at them from far away (so they appear smaller). The problem appears to be that your brain is constantly seeing the ‘normal’ world all around, and that apparently ‘overrides’ many perceptions or behaviors you might otherwise have.”

Welch further explains that in demonstrations we see today with HoloLens or Apple ARKit, for example, virtual objects can appear to be fixed on a flat surface, but beyond the basic shape and visual appearance, the software usually doesn’t recognize many important physical characteristics of the object, such as weight, center of mass, and behaviors, or the surface it’s on—much less any idea about the real-world activities occurring around the objects. “If I somehow roll a pair of dice on a virtual table, it will likely not ‘fall’ when it reaches the edge, and surely won’t bounce according to its type and the material of the floor,” he explains.

In a paper that Welch coauthored with Professor Jeremy Bailenson, the director of Stanford University’s Virtual Human Interaction Lab (VHIL), they outline some of the results of their research, which shows how virtual content is much more impactful when it behaves in the way we would expect physical objects to behave in the real world.

“In my lab, we are starting to use the HoloLens to understand the relationship between AR [augmented reality] experiences and subsequent psychological attitudes and behaviors toward the physical space itself,” Bailenson says. For example, he explains that his experiments indicated that virtual humans who “ghosted” through real objects—i.e., passed through them rather than going around or avoiding them as you would in the real world—were perceived as less “real” than ones who visibly obeyed the laws of physics.

Advances in mixed reality are likely to bring us headsets that are increasingly affordable and lightweight, but it is also possible that at least some of our future interaction with this technology will not involve wearables at all. “Spatial Augmented Reality” (SAR), for instance, which Welch developed with colleagues years ago, allows you to use projectors to change the appearance of physical objects around you, such as a table’s material or the color of a couch—without glasses.

“Of course SAR won’t work for all situations, but when it works, it’s really compelling and liberating,” Welch says. “There is something magical about the world around you changing when you don’t have anything to do with it—no head-worn display, no phone, nothing. You just exist in a physical world that is changing virtually around you.”

A Virtual Tool For A Collaborative Real World

Nonny de la Peña, founder and CEO of the immersive media company Emblematic, helped pioneer the use of virtual reality as a reporting and storytelling medium. Known as the “godmother of VR,” she believes immersive technologies are the closest thing to giving audiences “the view from the ground”—i.e., putting them right on the scene of a journalist’s report as it’s unfolding. She sees the HoloLens as having the potential to increase the quality and depth of our understanding of the world, particularly with the volumetric capture technique, which creates a 3D model of subjects via multiple cameras and green screen.

“Microsoft started offering high levels of realism using volumetric capture, something that’s just becoming part of the journalism tool set,” de la Peña says. Emblematic’s own After Solitary, an award-winning documentary produced in partnership with PBS and the Knight Foundation, used this technique to give the audience a more visceral sense of the psychological trauma of long-term imprisonment.

The main shift that mixed reality promises to bring about is that content will not be anchored to any one particular device. MR uses building blocks (real-world objects or computer-generated ones) to create environments that people enter and use to interact with one another. In that context, devices become a window that allows you to look into and access those worlds, rather than a repository where your personal content lives (like your smartphone).

---

Advances in mixed reality are likely to bring us headsets that are increasingly affordable and lightweight
Share on X

---

---

This has profound implications for how we will design apps in the future
Share on X

---

Kipman points out that in these shared real/virtual environments, our relationship with computing changes from a personal to a collaborative one—from devices storing your own individual content, to common creative spaces mediated by technology.

This has profound implications for how we will design apps in the future, according to Kipman. If, for example, you create a virtual statue and place it as a hologram on top of a table in your living room, another person with a different mixed reality device should still be able to see your statue when they enter that room and move it around if they wish. That’s because the device does not store your content, but rather scans and maps the environment to determine what objects (both real and virtual) inhabit it.

“These concepts require you to redefine an operating system in the context of mixed reality,” Kipman says. “You have to build a foundation that goes from the silicon to the cloud architecture that enables this shift from personal computing to collaborative computing. And these things take time.” Kipman smiles. “Until it doesn’t, then it just picks up and you’re like, What happened?”

This article was first published on the Fast Company Website

Is mixed reality the future of computing? https://t.co/CtDRqiReit pic.twitter.com/zyyU3kZIrS

— Fast Company (@FastCompany) October 6, 2017

If you would like to find out how to leverage VR/AR/MR in your enterprise, Tech Trends offers Virtual Reality Consultancy support

Alice Bonasio is a VR Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio and @techtrends_tech on Twitter.

The post How Mixed Reality Will Reinvent Computing appeared first on Tech Trends.

At #MSBuild2017 Microsoft’s Alex Kipman defined the company’s Mixed Reality vision for developing what he called the “Virtual Interaction Spectrum”  

I first got in touch with Alex Kipman back in 2008, when I was writing an article for a gaming magazine about a secretively code-named “Project Natal,” which later became the Kinect motion sensing system for the Xbox 360. For me, it was interesting to see how this rising star at Microsoft – a Brazilian expat like myself – was finding new and intuitive ways for humans and machines to interact. And although the focus of that technology was ostensibly on gaming, one couldn’t fail to spot broader potential applications down the line if they pulled this off.

 

Alex Kipman sporting the new MR headset

 

In this Huffington Post article I look at the recent announcements the company has made and how that ties into their long term vision for the development of Mixed Reality Technology

Mixed Reality: Can Microsoft Build IT? https://t.co/slUpTKOje8 via @HuffPostBlog

— Alice Bonasio (@alicebonasio) May 17, 2017

For companies looking to gain a competitive edge through technology, Tech Trends offers strategic Virtual Reality and Digital Transformation Consultancy services tailored to your brand.

Alice Bonasio is a VR and Digital Transformation Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio on Twitter.

The post Microsoft’s Mixed Reality Play appeared first on Tech Trends.