#MSBuild2017 Archives - Tech Trends

Microsoft’s Mixed Reality Play

Alice — Mon, 29 May 2017 09:46:33 +0000

At #MSBuild2017 Microsoft’s Alex Kipman defined the company’s Mixed Reality vision for developing what he called the “Virtual Interaction Spectrum”

I first got in touch with Alex Kipman back in 2008, when I was writing an article for a gaming magazine about a secretively code-named “Project Natal,” which later became the Kinect motion sensing system for the Xbox 360. For me, it was interesting to see how this rising star at Microsoft – a Brazilian expat like myself – was finding new and intuitive ways for humans and machines to interact. And although the focus of that technology was ostensibly on gaming, one couldn’t fail to spot broader potential applications down the line if they pulled this off.

Alex Kipman sporting the new MR headset

Fast-forward to Microsoft Build 2017, and Kipman is something of a rock star figure, cheered enthusiastically as he took to the main stage for his day 2 keynote on Mixed Reality. Because if Microsoft now believes that MR represents “the new frontier of computing”, it’s in large part down to him pushing and developing that vision over the past decade. He is ultimately working towards a world where we can interact with computers in as natural a way as possible, and all devices eventually become lenses.

The terms Mixed Reality is often dismissed as a marketing ploy
Share on X

In his keynote Kipman also took the opportunity to address the sometimes controversial terminology surrounding that vision, referring to a “Mixed Reality Spectrum”, which goes all the way from simpler forms of augmented reality (think Pokémon Go) to fully immersive virtual reality to 3-D interactive holograms. It boils down to thinking about the interaction between the virtual and real worlds in terms of “and” instead of “or,” he said.

It makes sense to think of virtual interactions as part of a spectrum
Share on X

In keeping with that strategy, Microsoft unveiled hardware and platform features designed to support the growth of what it now officially calls “Windows Mixed Reality.” These moves are meant to enable developers to create compelling content that will in turn drive broader consumer adoption and treats all forms of Virtual and Augmented reality as part of this broader continuum.

And as much as some sceptical pundits might dismiss this terminology as a marketing play, it is actually based on a widely cited paper on the subject published by Paul Milgram. Even back in 1994 when the paper was first published, A Taxonomy of Mixed Reality Visual Displays already had difficulties in establishing clear-cut categories that differentiated between these different types of virtual experience, and predicted that in future they would increasingly blend into one another.

What people like Kipman argue is that different headsets and different experiences will mix the physical and virtual realities to varying degrees, and that each of those experiences will fall within a certain place within the Mixed Reality spectrum. We will increasingly have more versatile hardware and content which allows users to navigate through that spectrum seamlessly, so there will be no need to worry about which type of headset you have. The best possible experience will automatically be delivered to that HMD.

Mixed Reality devices will deliver the best experience to users automatically
Share on X

This context helps us to make sense of why Microsoft insists on calling the headsets unveiled at Build Mixed Reality as opposed to Virtual Reality – which is what most people would term the fully occluded experience they offer at this stage. Built in partnership with Acer and HP, these use the same groundbreaking inside-out tracking technology developed for the HoloLens. As of now developers can pre-order these in advance of a consumer launch scheduled for the 2017 holiday season. Additionally, Microsoft also plans on releasing a set of Mixed Reality motion controllers which – unlike systems like the HTC Vive and Oculus Rift – will not require any external physical markers, working instead with the headset’s internal motion sensors.

Microsoft's new headsets use the same inside-out technology as the HoloLens
Share on X

In this Huffington Post article I look at the recent announcements the company has made and how that ties into their long term vision for the development of Mixed Reality Technology

As much as the hardware tends to grab most of the attention at such events, Microsoft’s bigger play is definitely all about the platform on which such devices – and many more to come – will run
Share on X

A hint of how that integration across devices works was showcased during the Cirque du Soleil demo (more on that below), where an avatar was remotely beamed in to join the HoloLens-wearing people on stage. That person was able to see what they were seeing and talk to them, using one of the new fully occluded headsets. The idea is therefore for people to be able to mix and match HoloLens with headsets such as the new Acer one as developers design apps where they both can share experiences in Mixed Reality. The “shell” of the user interface would of course vary depending on the display technology, but each experience would be running on the same software. And Microsoft wants that software to be Windows 10, which is why it has chosen to focus on platforms rather than devices. Instead of getting too hung up on defining and developing for each subset of the Mixed Reality continuum, they see the advantage in taking a holistic approach.

Yet as much as the hardware tends to grab most of the attention at such events, Microsoft’s bigger play is definitely all about the platform on which such devices – and many more to come – will run. The Windows 10 Creators Update began rolling out last month to more than 500 million Windows 10 devices around the world, and the Fall Creators Update will follow later this year. It incorporates a new Fluent Design System that promises to deliver an intuitive, harmonious, responsive and inclusive cross-device experiences and interactions.

Getting things done across all your devices should be easy, and ultimately not something you need to think about at all
Share on X

What all that blurb translates into is the idea that getting things done across all your devices should be easy, and ultimately not something you need to think about at all. The user experience should flow seamlessly between Windows, iOS, and Android devices, and as more Mixed Reality-enabled devices join that ecosystem, that will become a bigger part of that content mix. One of the most popular demos at Build – the Story Remix app – showed how that looks in practice, leveraging the Microsoft Graph to create stories with a soundtrack, theme, and cinematic transitions as well as the ability to incorporate MR elements, adding 3D objects to photos and videos.

The Story Remix app leveraged the Microsoft Graph to create stories with a soundtrack, theme, and cinematic transitions
Share on X

A team from Cirque du Soleil demonstrated a scenographic Mixed Reality tool developed for them by Finger Food Studios, a Microsoft HoloLens partner based near Vancouver
Share on X

But as impressive as the Story Remix demo was, the pièce de résistance was brought to the Seattle audience courtesy of a bunch of creative Canadians. A team from Cirque du Soleil demonstrated a scenographic Mixed Reality tool developed for them by Finger Food Studios, a Microsoft HoloLens partner based near Vancouver. It enables the creation of interactive 3D blocks and shapes that can be quickly transformed into a full scale rendering of a stage. It’s designed to help creators make faster and better decisions around casting, costume, stage height and much more by giving them the ability to test out ideas at scale before even putting out a casting call.

What set this demo a notch above the rest was the addition of live performers, conjured up just as they would in a live Cirque du Soleil show
Share on X

Ryan Peterson, Finger Food’s CEO, said they managed to turn around in just three weeks, drawing upon their experience of producing large-scale holograms (having previously worked with truck manufacturers Paccar to improve the efficiency of its industrial design processes).

Yet while the manipulation and scaling of objects was good, what set this demo a notch above the rest – as befitted a keynote unveiling – was the addition of live performers, conjured up just as they would in a live Cirque du Soleil show; The contortionists and dancers gave what would have been an impressive technical demo a “One More Thing” kind of edge. This felt like the start of something big.

Later on, I asked Bernard Fouché, General Manager, Innovation, at Cirque du Soleil’s C:Lab how they got the moving holograms of the performers to look so good:

“A couple of weeks ago the Cirque just happened to be performing in Seattle, so they asked the performers to go the Microsoft campus in Redmond so we could hologram them,” he revealed.

Their overarching goal is always to use technology to support the creative process, explains Fouché. Putting together a Cirque du Soleil production is a process which takes almost two years, and they hope that by using HoloLens they will not only be able to better communicate their plans and vision with investors and partners at an earlier stage, but also avoid costly mistakes which only become apparent when seeing things in full 3-D scale. “That’s why our studio in Montreal is the exact same scale as one of our big top shows. Scale is extremely important as creative people are very visual.”

Another use case where the advantage of using Mixed Reality becomes apparent is data visualization. My demo of new Acer headsets at Build, for example, was done in collaboration with Datascape, and allowed me to quickly and easily wrap my head around complex datasets which mapped out weather conditions in such a way as to allow me to make fairly informed decisions about where to best place solar panels and wind turbines within minutes. It’s hard to argue that making this sort of thing available to every policymaker – and voter – out there wouldn’t be an incredibly valuable and powerful proposition.

Another use case where the advantage of using Mixed Reality becomes apparent is data visualization
Share on X

The challenge is to bring the power of vision into people’s everyday lives
Share on X

For certain kinds of data there is even more of an advantage, as is the case with Earthquake data. Since earthquakes happen just below the surface of the earth at varying depths, even looking at 3D visualization of that data on a 2D screen it is difficult to make sense of patterns. Yet the exact same data presented on a device such as the HoloLens – where you can place a globe in the middle of the room and walk around, it, instantaneously allows users to spot the position of seismic faults and gives you a distinct advantage in spotting patterns that you might otherwise miss.

According to Lili Cheng, a distinguished engineer, general manager FUSE labs, the challenge is to bring the power of vision into people’s everyday lives. She’s been working with Kipman’s team to figure out ways of combining 2D and 3D elements with AI to create meaningful mainstream experiences, including integration with Cortana.

“We know we need to design everything to work with rich voice interaction, because that’s such a big part of how you want to interact with the HoloLens,” she says.

There is a general feeling that a lot of elements – such as cloud capabilities, artificial intelligence and hardware development – are coming together much faster than predicted, and that the only limits to the experiences we’ll be able to create will be our own imagination. It feels like an exciting time to be involved in that ecosystem, a bit like the early days of the Internet or Mobile, but even bigger in a way, since it will be an evolution of those technologies rather than replacing them.

Kipman’s excitement for what Mixed Reality will achieve is apparent, even after all these years
Share on X

Kipman’s excitement for what Mixed Reality will achieve is apparent, even after all these years. After the keynote he wanders the show floor, trying out all the different demos, in fact while I was waiting my turn at the Viacom NEXT booth he was so enthusiastic to see their Withdrawal music video demo (which both of us found to be pretty awesome) that he inadvertently cut in front of me in the line. After he’s finished, I jokingly berate him for it and charge him a selfie, exchanging a few words in Portuguese along the way. After years living abroad, it’s always nice to see a fellow Brazilian helping to shape the future.

It’s always nice to see a fellow Brazilian helping to shape the future.
Share on X

Mixed Reality: Can Microsoft Build IT? https://t.co/slUpTKOje8 via @HuffPostBlog

— Alice Bonasio (@alicebonasio) May 17, 2017

For companies looking to gain a competitive edge through technology, Tech Trends offers strategic Virtual Reality and Digital Transformation Consultancy services tailored to your brand.

Alice Bonasio is a VR and Digital Transformation Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio on Twitter.

The post Microsoft’s Mixed Reality Play appeared first on Tech Trends.

Sparking Creativity with AI

Alice — Wed, 17 May 2017 09:51:35 +0000

At Microsoft Build 2017 there was a focus on Artificial Intelligence as an enabler of widespread creativity across all industries.

It is hardly surprising that Artificial Intelligence was a major focus of Microsoft’s Build 2017 conference. In fact, given the rapid advancements in all areas of AI technology and the raging debates about if – or rather, when – robots will take over our jobs, it would be surprising if one of the world’s biggest technology companies weren’t thinking about these problems in a big way.

AI has the potential to disturb every industry
Share on X

When Harry Shum – Executive Vice President of Microsoft’s AI and Research group – took to the main stage on Monday, he talked about how soon it will be almost impossible to imagine a technology that doesn’t tap into the power of AI in one way or another. What made this possible, he continued, was the convergence of three forces – increased cloud computing power, algorithms running off deep neural networks, and access to massive datasets. This means that AI does indeed have the potential to disturb every single industry and process out there.

Microsoft wants to use AI to amplify human ingenuity
Share on X

But while the disruption does seem inevitable, companies like Microsoft are betting they can make it a positive one, talking about the possibilities it brings to amplify human ingenuity, augmenting people’s capabilities and helping them to be more productive.

“I’m really glad we’re having all these conversations about the disruptive power of AI, and it’s a good thing that we’re being so thoughtful about the ways in which we’re designing these systems, because it’s crucial that they are designed for people, to help them do their jobs better,” says Lili Cheng distinguished engineer, general manager FUSE labs.

Microsoft has been creating building blocks for the current wave of AI breakthroughs for more than two decades, and now it wants to leverage that accumulated research with the massive amounts of data available to them through the Microsoft Graph portfolio of products and Azure’s computing power, moving towards its vision of the “Intelligent Cloud”.

“One of the things we’ve done with our bot framework and the cognitive services is that we componentized things so that they can be reused into other context. For Cognitive Services there are 29 and you can choose whichever one(s) you choose,” explains Cheng.

So by adding a few lines of code, developers can mix, match and customize AI functionalities to suit their needs, spanning functions such as translation, video deconstruction and search, gesture recognition and real-time captioning.

Broadly speaking, therefore, Cognitive Services are plugin functionalities that developers can use to enable systems within their apps to hear, speak, understand and interpret human needs. For example, LUIS (Language Understanding Intelligent Service) helps developers to integrate language models to understand users using either prebuilt or customized models. While the Custom Vision Service makes it easy to create your own image recognition service.

Microsoft now wants to redefine itself with AI by infusing it into every product and service it offers, from Xbox to Windows, from Bing to Office
Share on X

The overarching idea is to move towards what the company calls “conversational AI,” (where instead of humans having to learn about computers, computers will be able to understand humans) and thus enable more fluid and natural interactions between humans and machines.

Clearly when one thinks of conversational AI this includes Cortana – Microsoft’s personal intelligent assistant which is already used by 145 million people across the world, and the company is certainly focusing heavily on making it easier for developers to reach this growing audience by working in partnership with companies like HP, Intel and Harman on IoT devices such as Invoke intelligent speaker.

However, this is an extension of the Cognitive Services offering the company launched back in 2015, which has since attracted over 568,000 developers from more than 60 countries to sign up. Microsoft now wants to redefine itself with AI by infusing it into every product and service it offers, from Xbox to Windows, from Bing to Office.

Cognitive Services are plugin functionalities that developers can use to enable their apps to hear, speak, understand and interpret human needs
Share on X

AI has the potential to become a much more creative tool, not something that can only be leveraged and understood by Data Scientists
Share on X

An example of this was the demo showcased during the keynote which showed how Microsoft’s translation APIs could provide real-time translation in multiple languages during any PowerPoint presentation. It not only embeds live transcriptions of the presenter into a specific language, but also generates an unique link which lets attendees get further live translations into their own preferred languages – all in real time. Other such functionalities include Project Prague – an early stage SDK that matches captured user movement against an extensive library of hand poses, allowing the developer to tie app actions to gestures and create a more intuitive user experience through gesture interaction.

The ease with which developers can incorporate these elements into their apps, together with this growing emphasis on natural interaction, means that AI has the potential to become a much more creative tool across the board, not something that can only be leveraged and understood by Data Scientists. This democratization of AI is in fact a cornerstone of Microsoft’s vision, and something Cheng is very keen to enable:

“You need to make people laugh, and you need creativity and you need all this stuff because that’s what being human is about. This better be a creator platform for writers and other non-technical people. AI can’t be just for people with PhDs we want people who are writers, artists or builders to be able to use those tools to get their jobs done better. We’re super committed to that. There’s a lot more to AI than writing code, and I think that has to be our goal, to make people want to make the things they live in all day,” she concludes.

This article was originally published on The Next Web

How Microsoft wants us all to get creative with Artificial Intelligence https://t.co/3phsuZvZX0

— Alice Bonasio (@alicebonasio) May 16, 2017

Alice Bonasio is a VR and Digital Transformation Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio on Twitter.

The post Sparking Creativity with AI appeared first on Tech Trends.

New VR and MR Hardware Unveiled at Microsoft Build 2017

Alice — Thu, 11 May 2017 16:36:04 +0000

Microsoft announces Windows Mixed Reality Dev Kit and world’s first MR motion controllers with no markers required at #MSBuild2017

Microsoft is looking to expand its Mixed Reality offering with new hardware and tools
Share on X

The Windows 10 Fall Creators Update demoed today at the Build Developers Conference in Seattle focuses on the concept of Fluent Design which wants to bring Mixed Reality to consumers around the world.

Using just the sensors in the headset, the motion controllers offer responsive movement tracking
Share on X

We’ll go deeper into what those features look like in our Day 2 update – but we were so excited about the freshly unveiled VR and MR hardware that we thought it deserved an update all of its own.

“We are working with developers to create this new frontier in computing and want any developer to get a Windows Mixed Reality dev kit.”

From today, developers in the US and Canada will be able to pre-order an Acer ($299) or HP ($329) headset from the Microsoft Store for delivery later this summer.

Microsoft unveiled Mixed Reality motion controllers that don't need physical markers in the room
Share on X

They also introduced the world’s first Windows Mixed Reality motion controllers which does not require markers. This means that – unlike a headset like the HTC Vive where you have to set several sensors around the room to allow the headset to accurately map where you are in relation to the VR/MR elements. The motion controllers offer precise and responsive tracking of movement in your field of view using just the sensors in your headset.

“A customer who pairs a Windows Mixed Reality headset with motion controllers will have a rich and immersive experience across creativity tools, productivity, games, and entertainment. There is no need to install hardware on the walls around you. We created the controllers as a high quality and comfortable input device with the same ease of set up and portability as our headsets. “

The plan is to have a consumer-ready version ready for the 2017 holiday season

“We are keeping our promise to deliver a new mixed reality experience for everyone at an affordable price point. Today we announced Acer will sell a Windows Mixed Reality headset and motion controller bundle priced at $399 this holiday.”

Alice Bonasio is a VR and Digital Transformation Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio on Twitter.

The post New VR and MR Hardware Unveiled at Microsoft Build 2017 appeared first on Tech Trends.

#MSBuild2017 News and Announcements

Alice — Wed, 10 May 2017 21:37:35 +0000

Thousands of developers gather in Seattle as Microsoft Build unveils new products and an ambitious vision for the future of technology.

We're seeing a paradigm change in user experience
Share on X

The opening keynotes of this morning’s conference – from Microsoft CEO Satya Nadella and Executive Vice Presidents Scott Guthrie and Harry Shum – outlined the company’s vision for becoming the world’s largest productivity platform. With over 500 million Windows 10 monthly active devices, and 90% of Fortune 500 companies already using the Microsoft Cloud, their combined platforms offer developers like those in the audience “over a billion opportunities to reach and connect with Microsoft customers. You can, for example, achieve single sign-on across 12 million organizations, dramatically reducing friction, yet this fantastic opportunity comes with tremendous responsibility,” said Nadella. “The future of technology will be determined by the choices we make as developers.”

The future of technology will be determined by developers' choices
Share on X

He then showed the audience a poignant video which told the story of how Haiyan Zhang, Innovation Director at Microsoft Research Cambridge, invented and programmed a wearable that helped designer Emma Lawton manage the hand tremors caused by her advancing Parkinson’s disease. It ends with the moment when Lawton – wearing the Emma, the watch-like device that was named after her – managers to write her name and draw straight lines again for the first time in many months. As they cry openly, it was difficult not to join in, and as both women joined Nadella onstage they received the morning’s biggest round of applause.

Although the rest of the demos and testimonials weren’t quite as emotional as that, the common theme that ran through all of them was one of highlighting how all these high-level technologies actually translate into real-world problem solving for businesses, individuals, and society as a whole. The tone it set for the conference was one of grounded and confident optimism for the future that developers can build with them, and, that takes shape in the idea of the Microsoft Graph.

We're moving towards an intelligent cloud with an intelligent edge
Share on X

What underpins this expansion of the Microsoft Graph is the idea that we’re moving from a mobile-first and cloud-first world towards an integrated intelligent cloud with an intelligent edge. This intelligent cloud will ultimately enable us to search the real world in the same way as we do the digital one, by building in contextual awareness that combines insights from devices and the world of work.

“This is a fundamental change in the paradigm of the apps that we are building,” says Nadella. And because this new type of user experience that is emerging will span all our devices, it will need new platform capabilities such as Azure IoT Edge, which seamlessly extends cloud computing to a range of edge devices.

Artificial Intelligence is also a big piece of that vision, and Microsoft’s ambition is to use AI to augment the capabilities of every developer, organization, platform and person to achieve more, “amplifying human ingenuity with intelligent technology.”

It is ultimately up to us whether the future we build with technology becomes an utopia or a dystopia
Share on X

Ultimately, as Nadella concludes, it is up to us whether the future we build with technology is utopian or dystopian. He told the audience he was an unapologetic technology optimist, yet also keenly aware of the unintended consequences that often come from adopting new technologies. In other words, we need to guard against an Orwellian 1984 scenario while embracing the fact that we’re very much living in a Brave New World.

The user experience of the future will span all your devices. From mobile first to the intelligent cloud @satyanadella #MSBuild2017 pic.twitter.com/WxXD0CHvWL

— Alice Bonasio (@alicebonasio) May 10, 2017

Here are some of the main announcements and news from Microsoft Build 2017

Azure Cosmos DB – Touted as the industry’s first globally distributed multi-model database services built from the ground up to power planet-scale cloud services and data-intensive applications — from IoT to AI to mobile — it promises greater fault tolerance and support for all data types, delivering millisecond latency at the 99^thpercentile
Visual Studio 2017 for Mac enables developers to work seamlessly across Windows and Mac environments with full support for mobile, web and cloud workloads, and previews of Docker tools, Azure Functions and Xamarin.IoT support.
Customizable cognitive services such as Bing Custom Search, Custom Vision Service, Custom Decision Service and Video Indexes now join a roster of 29 services available to developers, each with unique customization options that enable customization of intelligence capabilities such as vision, speech, language, knowledge and search into apps and bot experiences.
Cognitive Services Labs were also launched, enabling developers to experiment with new services, such as a Gesture API, which, although still in early stages, offers some exciting possibilities for developing Mixed Reality applications for devices such as the HoloLens, which are already operated partially through gestures
Cortana Partnerships were signed with HP on devices and Intel on reference platforms to deliver Cortana-enabled devices. Additionally, the Cortana skills kit is now in public preview (Currently S. only), where developers can build skills for Cortana by creating a bot and publishing it to the new Cortana channel of the Bot Framework across Windows 10, Android, iOS and the new Cortana-powered Harman Kardon Invoke speaker.
MySQL and PostgreSQL-managed services that join Azure SQL Database to give developers expanded choice and flexibility on a service platform to deliver more availability and scalability, with minimal downtime, plus data retention and recovery.
Database migration services, which will allow Oracle and SQL Server customers to more easily move their data and quickly modernize their apps..
Windows Server Containers support in Azure Service Fabric, with Visual Studio tooling, and a preview of the ability to use Docker Compose support for Service Fabric to deploy containerized apps to Service Fabric — enabling developers to deliver mission-critical, scalable apps and services.
DocuSign supports a rapidly growing customer base of 300,000 companies and more than 200 million users across 188 countries. The company today announced Microsoft Azure is its preferred cloud for global expansion, starting in Canada. DocuSign uses Azure SQL Database to process much larger volumes of digital transactions.
Microsoft Graph APIs available to developers, including APIs from SharePoint and Planner. The Microsoft Graph gives developers access to Office 365 data and intelligence and helps connect between people, conversations, projects, schedules, processes and content. These insights help developers build smarter apps and enable smarter ways to work.
Presentation Translator, which leverages Microsoft’s Translation APIs, was featured, allowing real-time translation to multiple languages during any presentation.
Adaptive cards supported by the Microsoft Bot Framework, developers can write cards once that work across multiple apps and platforms. Using the Bot Framework, developers can also now publish to new channels including Bing, Cortana and Skype for Business, and implement Microsoft’s payment request API for checkout in their bots.
Azure Batch AI Training is a new Azure offering currently only available in private preview, that will allow developers and data scientists to configure an environment with parameters and run their models against multiple CPUs and GPUs, and eventually field-programmable gate arrays.
Tact – the sales experience platform that turns any connected device into a AI-powered virtual sales assistant was demoed in anticipation of its release later this year, showing how multiple Microsoft products such as Dynamics 365, Office 365, Microsoft Teams, Cortana Skills, Microsoft Graph and Sentiment Analysis, will be integrated into it.

Alice Bonasio is a VR and Digital Transformation Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio on Twitter.

The post #MSBuild2017 News and Announcements appeared first on Tech Trends.