Why would we want to build a parallel reality populated by digital twins?

Sure, it sounds cool, yet this “virtual twinning” effectively extends the possibilities of both the “world of atoms and the world of bits“ as HoloLens inventor Alex Kipman eloquently put it in a TED talk which has been viewed over 3.5 million times.

There is already an entire ecosystem emerging around the challenges of sensing, mapping, and replicating objects from the real world into the digital – and vice-versa
Share on X

This is not some distant blue-sky thinking. There is already an entire ecosystem emerging around the challenges of sensing, mapping, and replicating objects from the real world into the digital – and vice-versa. In fact, one of the main applications of the 5G connectivity infrastructure rollout will be to mirror physical objects and environments digitally in real-time. That, in turn, would enable XR (extended reality) applications to be built which bring quantifiable value to users.

5G applications that consumers will “get”

“I don’t think the average person really comes close to realizing just how much of a quantum leap we’re talking about,” says Deeyook CEO Gideon Rottem. “They think it will be faster, but literally it will enable things that will make what we have now pale in comparison,” he adds.

Rottem’s Israel-based company has developed a self-learning tracking solution that measures the angles of wireless transmissions – tapping into the existing 1.7 billion wireless access points that already exist globally – to determine positioning of employees, items and assets (within 10 cm) without tapping into a company’s IT network and without invasive personal tracking or data capturing.

These mapping technologies are going to interact with the cloud differently, as it no longer becomes necessary to upload images in order to understand what the user is standing in front of. According to Rottem, once we achieve accurate positioning, we can also create accurate virtual worlds (largely automatically) so the integration between the physical and the digital can become seamless.

These mapping technologies are going to interact with the cloud differently, as it no longer becomes necessary to upload images in order to understand what the user is standing in front of
Share on X

A newly released report by research and consulting firm Strategy Analytics revealed that although consumers are intrigued by 5G, speed on its own is not enough to sell them on the idea of upgrading. This is partly because it is difficult to grasp what it will enable them to do. CSPs will need to solve existing pain-points and entice customers with new experiences.

In practical terms, spatial mapping technologies are crucial in enabling the type of integrated applications that not only address practical customer needs but also create these “wow” factors that will prompt mass adoption of 5G. The above research found that what got consumers most excited about the prospect were immersive experiences such as holographic conversations and XR field training.

A newly released report by research and consulting firm Strategy Analytics revealed that although consumers are intrigued by 5G, speed on its own is not enough to sell them on the idea of upgrading
Share on X

It’s not a surprise, therefore, to see many companies in the connectivity space embracing immersive technology and content as they seek to bridge this “imagination gap”.

An example of a CSP doing just that is UK mobile giant Three, which recently released a highly produced tongue-in-cheek advert which shows us what a 5G-enabled holographic future could look like.

“Hold on to your crumpets,” says Three in their recent 5G ads.

Many companies in the connectivity space embracing immersive technology and content as they seek to bridge the consumer imagination gap
Share on X

Moetsi is a company working to build the spatial mapping infrastructure to make that future happen by offering “sensor stream processing on the edge.” This effectively boils down to processing vast amounts of data in real-time and delivering it in a format that makes sense and is relevant to the context of where it is being applied.

We will see an explosion of XR applications that provide immense value industry and enterprise, as well as mind-blowing experiences for individual consumers
Share on X

To illustrate how an application could leverage this, Moetsi co-founder Adam Polak gives an example of someone about to cross a street getting alerted that the car moving towards them isn’t slowing down for the red light. At that point, all our decisions would potentially become augmented and informed by tapping into these edge-processed datastreams, fundamentally changing our relationship with technology, the world around us, and one another.

3D mapping in arts and entertainment

There are use cases already emerging of spatial mapping being applied to create, transform, and augment art. Award-winning immersive content studio HEAVY has partnered with the Marriott hotel group to build a “secret digital garden” at the JW Marriott Anaheim Resort which was  due to open in April. The project leveraged mapping technology from 6D.ai, a company that set out to build  “a 3D map of the world” and has since been acquired by Niantic, the makers of Pokémon GO.

This feature of persistence is key to understanding the value these real-time mapping technologies bring to such experiences. In this case, HEAVY produced a fine art installation blending physical, custom-made sculptures with augmented reality elements to create a serene space where resort guests and visitors can plant their own fantastical trees while enjoying playing among digital caterpillars and butterflies.

There are use cases already emerging of spatial mapping being applied to create, transform, and augment art
Share on X

What makes the zen-like experience more than a bit of fun, however, is the narrative backdrop for the magical plant’s evolution from sapling to tree. By performing the digital meditations, each guest will figuratively transform into a different kind of tree, personalized around their choices, which can then be further nurtured over time like a flora version of Tamagotchi. This means that the way users interact with and nurture their trees will affect how it develops, and consequently what type of experience each person generates and enjoys.

How will this technology come to life?

Improving the speed at which connected devices can stream information to an authoritative server will be a crucial piece of that puzzle according to Polak, as improvements in networking such as 5G will help reduce the latency and increase the amount of sensors that can communicate in a local area, something that the current generation of networking infrastructure simply cannot cope with.

“The engineering challenge is enormous: being able to interpret any sensor data correctly in a manner that can be used by application developers to provide value to end-users, basically not only seeing, but understanding anything a device can sense in the world,” Polak adds.”

So, are we there yet? Well, yes and no. A lot of this technology already exists but it is hampered by the fact that nearly every XR developer has to roll out their own solutions to build cross-platform applications that can be shared with all users.

According to Polak, initiatives like OpenXR are addressing the problem of having to build separate implementations for ARKit, ARCore, and MRTK (Microsoft’s Mixed Reality Toolkit) by enabling developers to access XR device data through a single interface.

A lot of this technology already exists but it is hampered by the fact that nearly every XR developer has to roll out their own solutions to build cross-platform applications that can be shared with all users
Share on X

Once we arrive at that convergence of 5G connectivity and the ability to interact with digitized objects in platform-agnostic experiences that are embedded in our own physical reality, our world as we know it will never be the same. We will see an explosion of XR applications that provide immense value industry and enterprise, as well as mind-blowing experiences for individual consumers.

So while wrapping your head around the concept of building a “Mirror World” might be difficult at first, once it clicks into place we’re likely to wonder how we ever settled for such an ordinary reality in the first place.

This article was originally published on Futurithmic

#SpatialMapping technologies are crucial in enabling the type of integrated applications that not only address practical customer needs but also create these WOW factors that will prompt mass adoption of #5G. https://t.co/2wC9FhIFdq pic.twitter.com/vnsLln2VPY

— Futurithmic (@futurithmic) May 1, 2020

Tech Trends offers a broad range of Digital Consultancy services to guide companies, individuals, and brands in effectively leveraging existing and emerging technologies in their business strategy.

Alice Bonasio is a XR and Digital Transformation Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio on Twitter.

The post Digitizing our Existence appeared first on Tech Trends.

By Moetsi.

With Sturfee’s AR cloud technology just out of stealth mode, Tech Trends spoke to the company’s CEO, Anil Cheriyadat, and Head of Business Operations & Partnerships, Sheng Huang, about their exciting new tech and what it’s going to take to digitize the world.

Just recently out of stealth mode, Sturfee wants to digitize the world around us with its AR mapping technology
Share on X

ANIL

Before starting Sturfee, I worked at the Oak Ridge national lab, which was home to Summit – the fastest computer in the world at the time, working on computer vision and AI algorithms to be used on satellite imagery in real-time. Really, this was the only place in the world you could do that sort of thing then. And it was here that I developed a passion for understanding the world through imagery — and where Sturfee, as an idea, really began.

I developed a passion for understanding the world through imagery, and that's where the idea for Sturfee was born
Share on X

SHENG

For me, I was running biz ops at Niantic right at the beginning. I was there when they launched their very first app — the field trip app — which I helped scale from a few thousand users to a few hundred thousand and eventually millions of users. It was incredible to see the uptake and the real potential of location-based AR gaming.

By creating an AR cloud there is the potential to revolutionize the way we interact with the world around us
Share on X

Which, of course, we can see from the incredible success of Niantic’s Pokemon Go and Harry Potter: Wizards Unite. But AR doesn’t begin and end with gaming. By creating an AR cloud there is the potential to revolutionize the way we interact with the world around us.

ANIL

Until now, cameras have been successfully trained to recognize human faces, but no one has built a similar technology to recognize physical outdoor locations. Now Sturfee has developed computer vision tech that can recognize places on the ground using images from satellites up in space. This will allow your phone’s camera to instantly recognize its surroundings and for applications to pull 3D building geometry and ground plane data.

Combining satellite imagery with mobile positioning, Sturfee allows developers to create a range of applications that will change the way you use your phone for navigation, tourism, ride-sharing, gaming, outdoor advertising, and marketing.

What astounded me was the accuracy to which modern satellites could create an image of the world. And how quickly too.

Sturfee has developed computer vision tech that can recognize places on the ground using images from satellites up in space
Share on X

ANIL

What most people don’t know is that satellite imaging has improved a lot in the last few years. New satellites can now pick up anything bigger than 30cm in the highest resolution and can create an image of the whole world in just 5-7 days in lower resolution. That’s pretty astounding.

What astounded me was the accuracy to which modern satellites could create an image of the world. And how quickly too
Share on X

That means that Sturfee’s AR mesh can pinpoint locations to incredible precision, and access a pretty up-to-date map of urban topology — buildings and roads tend not to change too much in 5 days. So with access to such cool satellite imagery, why did they turn their hand to AR?

Combining satellite imagery with mobile positioning, Sturfee allows developers to create a range of applications that will change the way you use your phone
Share on X

ANIL

Augmenting the world around us is fascinating. It creates a whole new layer of interaction with everything. You can create a digital overlay of information wherever you go, and what you can do with that is practically limitless. You can leave notes for your friends to discover in live locations, bookmark the world and play basketball on skyscrapers. It’s really cool.

Sturfee is attempting to solve the problem of creating a digital twin of the world at scale
Share on X

But this digital overlay is only possible if the real world is being scanned constantly. We need a true representation of the world, as well as the ability to store this map, in order to actually create these next-generation applications.

And it’s here that Sturfee is making a difference. Offering a Visual Positioning System (VPS) cloud service, Sturfee’s AR City platform is a major step towards understanding 3D locations from traditional GPS 2D mappings. And attempts to solve the problem of creating a digital twin of the world at scale.

You can leave notes for your friends to discover in live locations, bookmark the world and play basketball on skyscrapers
Share on X

ANIL

Building persistent replicas of large cityscapes is too computationally intensive to do from the “bottom-up” — having phones on the ground scanning their surrounding environments and uploading those scans to the AR Cloud. To solve this, we are coming from the “top-down”, which cuts both processing and financial costs. With our technology, we can create a machine-readable version of San Francisco in just a week, and detect and update any city changes even quicker.

SHENG

Right now Sturfee already powers 15 cities spread around the world, and we can move into a new city, have it scanned and ready in just a week, which is extremely quick. Mike Levine, CEO of HappyGiant, an AR app developer and publisher, worked closely with Sturfee to create this San Francisco demo using the Sturfee VPS platform in order to show off Sturfee’s city scale AR.

But don’t they also want to access bottom-up imagery too?

Right now Sturfee already powers 15 cities spread around the world, and we can move into a new city, have it scanned and ready in just a week
Share on X

ANIL

Yes, we definitely want to make sure our 3D meshes of environments are always up to date. That’s why we work with 3rd party providers to provide us with bottom-up imagery from “edge device” scans [an iPhone, for instance]. We combine this information from the two different perspectives into our pipeline to have the most robust digital representation of the world – from top-down, to bottom-up. We just want to be the most robust end-to-end computer vision pipeline.

Devices are simply not able to process a great deal of depth data, and so there’s not much of it
Share on X

And that’s always going to be the limit. Simply, satellites will not be able to get inside buildings any time soon and so Sturfee will need to rely on handheld tech to provide hyperlocal data to their machines. But this data is limited. Devices are simply not able to process a great deal of depth data, and so there’s not much of it.

This is where solutions like Moetsi’s Sensor Stream Pipeline come in; making it easier to process depth data by compressing it on device, and processing it off-device. Or 6d.ai, the leaders in crowdsourced 3D mapping using standard smartphones. But this does require a good data connection, something we’re all hoping 5G will improve

This is where solutions like Moetsi’s Sensor Stream Pipeline come in; making it easier to process depth data by compressing it on device, and processing it off-device
Share on X

Sheng

For devices to be able to send and receive information to each other really fast on the AR Cloud, the local network needs to be very fast. 5G will really help with this, of course. The information between edge devices will be sent so much quicker, which will really improve things like multi-player gaming experiences.

It’s no surprise then, that Sturfee has partnered with KDDI Corp, Japan’s second-largest cellular operator, to bring their VPS to all major metropolitan areas in the country. And with orders to integrate their tech with smartphones, wearables and 5G services on the horizon, it’s a really exciting time for the company, and to all of us looking forward to playing around in these Augmented Reality Cityscapes. 

Sturfee has partnered with KDDI Corp, Japan’s second-largest cellular operator, to bring their VPS to all major metropolitan areas in the country
Share on X

Olenka Polak is the co-founder of Moetsi along with Adam Polak, whose open source Sensor Stream Pipeline makes it easier to process sensor data data off device, without throttling bandwidth. When she’s not working on Moetsi’s open source project, you can find her at AWE nights in Brooklyn, NY, excited about all things XR.

*Answers have been edited for brevity and clarity

The post Making Cities AReady appeared first on Tech Trends.