In a recent PCMag article, Rob Marvin got hands-on with the Amazon Sumerian 3D development platform for building AR/VR apps and gives an in-depth overview of its features and the company’s broader Immersive Technology strategy.
One of Sumerian's biggest selling points is that despite how powerful the editor is, there's nothing to download or install. Click To TweetAccording to Marvin, Amazon’s decision to enter the AR/VR space with their Sumerian platform came down to a convergence of three factors: the emergence of smartphone-based augmented reality; untapped VR opportunities in the business-to-business (B2B) market; and helping AWS customers solve pain points with things they were already trying to do:
“The B2B applications include scenarios like interactive digital signage (think the giant talking hologram ads from Blade Runner: 2049), virtual training, and a host of industrial Internet of Things use cases, such as using sensors to create digital twins and complex simulations.The idea is that you have a high-quality, high-definition, context-aware sensor in the hands of billons of people.”
One of Sumerian’s biggest selling points, Marvin explains, is that despite how powerful the editor is, there’s nothing to download or install. During the demo, Sumerian loaded from a browser URL in seconds. Even doing real-time natural language processing (NLP) and rendering elaborate animations didn’t slow it down much.
“As with low-code development platforms, Sumerian can be used on a basic level with almost no developer experience. However, coders and data scientists can go a lot deeper with programmable APIs and Sumerian’s command-line interface to customize scenes and write complex app logic.”
In other words, the basic process for building a Sumerian app is the same regardless of the AR and VR platforms on which you ultimately publish it, you can jump straight into creating a new “scene” – default templates to choose from include office spaces, warehouses, a cargo ship, and a campfire.
“Sumerian plays not only in the 3D development space with platforms like Unity, Unreal Engine, and Vuforia, but also in the broader AR/VR ecosystem along with ARKit, ARCore, and Windows Mixed Reality. Between WebGL, WebVR, and WebXR, Sumerian is completely platform-agnostic, and Sumerian published native wrappers to integrate directly with ARKit and ARCore for smartphone-based AR apps. Sumerian can build apps for any platform that supports WebVR, meaning not only Oculus Rift and HTC Vive but also Samsung Gear VR, Google Daydream View, and others. Sumerian is also working closely with the Google Chrome team on WebXR for browser-based apps.”
Amazon’s strategy, much like Microsoft’s is to make it not only possible, but easy and attractive for everyone to integrate with their platforms and developing tools – The “build once, run anywhere” philosophy.
Amazon hopes Sumerian can play a part in spurring the industry to make 3D mass market products and drive down the cost Click To Tweet“What we decided instead is to be as broad as possible in supporting what we think will be a massive market. We’re going after the underlying foundation,” an Amazon spokesperson explained.
AI Hosts – 3D-animated characters you can place into an AR or VR scene – are another of Sumerian’s USPs. Users can ask hosts questions, and developers can script a complex set of actions, behaviours, gestures, and movements.
“With something called a point of interest system, you can check a box in the editor so the host’s eyes always pay attention to the camera. So if you’re wearing an HTC Vive Pro walking around a 360-degree space, the host can follow you. If it’s an AR app connected to your smartphone camera, Roche explained that Amazon’s Rekognition deep-learning system can run facial analysis of both where you are and where your face is in the frame to make it look like the host is looking back through your screen directly at you. It gives you the illusion of eye contact.”
Amazon is also scoping out ways to make it easier to create hosts, like using facial recognition to match rendered characters to real people. Additionally, there is a lot of work going on behind the scenes to integrate voice commands seamlessly into those experiences.
AI Hosts are one of Amazon Sumerian’s unique selling points Click To Tweet
“You can’t say “Hey Alexa” in a Sumerian app the same way you can activate Cortana within Windows Mixed Reality. Instead, Amazon uses the automatic speech recognition and natural language understanding APIs behind Alexa to let hosts have conversations… Imagine someone walking through a hotel lobby, a mall, or a stadium and seeing a host walking alongside them on a screen. It may sound a tad creepy, but hosts could also process contextual information like location to turn the ad into a personalized conversation.”
Amazon hopes Sumerian can play a part in spurring the industry to make 3D mass market products and drive down the cost. On the AR side, the basic enablers are in place thanks to ARKit and ARCore, and where it comes to VR, Amazon hopes to see hardware coming down in price, becoming less clunky to wear, and going wireless. According to their spokesperson:
“When that happens and you can wear a VR experience like a pair of glasses, VR will really take off. It needs to be as natural as watching a video on a tablet or a turning on a TV before it’s ready for mainstream consumption at the same level as the other screens we have today. Developing a whole ecosystem around it of content creators, advertisers, end-users, and companies catering to those users is how you do it.”
Another one of Amazon’s target use cases for Sumerian apps is education and training. Whether you’re learning how to use a medical device, service a vehicle, or learn a new language, it’s about dropping you in an environment that feels as visceral as possible like the real world.
Whether you’re learning how to use a medical device, service a vehicle, or learn a new language, it’s about dropping you in an environment that feels as visceral as possible like the real world Click To TweetYou can read the full article on the PCMag Website
Stop the Presses! @PCMag takes a look inside Amazon Sumerian, a web-based development platform for creating AR, VR & 3D applications. https://t.co/oktcAxOmI9 pic.twitter.com/Bi4Olth5fA
— Amazon Web Services (@awscloud) April 17, 2018
For companies looking to get into VR/AR/MR our Virtual Reality Consultancy services offer guidance on how these technologies can enhance and support your brand strategy.
Alice Bonasio is a VR Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio on Twitter
