Things like display, graphics, and rendering tend to get a lot of love in AR. One under-rated technical ability? Positioning. Accurate positioning helps to make sure that AR assets appear convincingly in the world. It also helps to ensure that assets tied to a specific geographic location appear where they are supposed to. Finally, it makes or breaks multi-user experiences.
This article will look at some of the companies rethinking positioning. We’ll also explore some of the technologies enabling these advances, and the AR experiences that these advances enable.
Some Technical Background
It’s important to understand at the outset that positioning and locating are not the same as “localization.” Locating and positioning are about understanding where a user is in the world. Localization is about convincingly placing a virtual asset in a user’s physical environment.
A lot of AR applications don’t need positioning or locating at all – they don’t care where you are in the world, but they need to be able to understand the world around you to place items. On the other hand, a lot of non-AR applications don’t need to know what the world around you looks like to help a delivery driver find you or help you find your next location.
However, some advanced AR applications need both. This is the case for AR activations that are tied to physical locations on the planet. It’s also one approach to multi-user AR experiences.
NextNav
Right now, most location systems work on the Global Positioning System – GPS. GPS can locate a device reasonably accurately for most use cases but it comes with a catch – GPS is flat. Humans, particularly in urban areas, tend not to limit themselves to the ground. We stack.
“I’m sure we’ve all experienced that, opening up Google Maps and there’s just this big blue dot,” said Dan Hight. “In the urban market, understanding your vertical location is important.”
Hight is VP of Business Development and Partnerships at NextNav. NextNav is a navigation company “elevating geolocation” by giving positioning data a “third axis.” You probably know that your phone has GPS, but NextNav uses a lesser-known sensor – the barometric pressure sensor that measures air pressure.
The higher up you go, the lower the barometric pressure gets. Barometric pressure also changes with the weather. So, NextNav uses information from a network of pressure stations, combined with information from your device’s barometric pressure sensor. The difference lets the system determine your elevation.
“If we know your location,” said Hight, “and we know the pressure, we can calculate your vertical position with floor-level accuracy.”
The AR Connection
NextNav was founded to work with emergency response – which is what GPS was designed for in the first place. And they do partner with emergency response groups around the country. But, they also recognize the benefits that their solution has for consumer applications, particularly in AR.
“AR, foundationally, is resolving itself today around recognition of the physical world,” said Hight. “AR is going to be greatly enhanced with metadata provided by better geolocation.”
Further, earlier this year, NextNav announced a partnership with echo3D. echo3D is a cloud platform for managing and delivering immersive experiences. With the partnership, creators could make such things as location-based AR experiences that are different on every floor of the same building.
“Precise location capabilities, including the vertical dimension, are foundational to the future of immersive mixed-reality experience,” echo3D co-founder and CEO, Alon Grinshpoon, said in a blog post announcing the partnership. “Alongside NextNav, we’ll continue providing creators with the capabilities needed to bring the promise of AR, VR, and the metaverse to life.”
Auki Labs
GPS isn’t just flat. It’s also kind of scary. GPS is a point of failure and it’s that dirtiest word of the twenty-first century: “centralized.”
“We happen to believe that a decentralized positioning system is better than a centralized positioning system,” said Nils Pihl. “Wherever there is an opportunity to make something better in human life, GPS just doesn’t cut it.”
Pihl is the founder and CEO of Auki Labs. A few years back, Auki Labs was trying to make a multiplayer AR tabletop experience. They realized that “shared AR was next to impossible,” but they also came to another conclusion.
“What we realized is that everyone is looking very much at this concept of digital twins and positioning yourself in the real world,” said Pihl. “For most of the things that people want to do with shared AR, it’s not important where you are in the world – it’s important where you are in relation to me.”
The end result: a system that locates users with relation to one another rather than with relation to the globe. The system has had successful demos with 50-500 simultaneous users. That’s significantly higher than experiences by Niantic and with a significantly shorter calibration time, according to Pihl.
The Problem of Persistence
Multi-user experiences are one of the high pursuits of AR. Another is persistence – the ability of an experience to exist in space when no one is using it. Auki Labs’ solution to mapping solves a lot of the problems with GPS, but GPS has its finger on persistent positioning. That doesn’t mean that Auki Labs doesn’t have its own approach.
“What we’re building out for persistence is not our own digital twin, but the ability for people to have their own digital twins that are interoperable,” said Pihl. “I would love to have a digital twin of my home, but I want it on my machine – and just because it’s on my machine doesn’t mean I don’t want other people to have access to it.”
The solution is beautiful but it’s a mouthful: “decentralized interoperable domains.” Each person makes their own maps and stores their own maps. When they want to have a multi-user experience, they choose to share those maps with the other users. The protocol is designed to be used on everything from modern mobile phones to AR glasses to self-driving cars.
Auki is working on other positioning and mapping solutions as well, including ultrawideband triangulation. This method locates devices using the strength of nearby UWB transmitters – potentially in lieu of visual inputs.
“It would be possible, in theory, to build AR glasses with no camera,” said Pihl. “We’re not a big surveillance capital company, so we’re not married to that business model.”
Orbbec 3D
For the most part, asset creation, localization, and body, hand, and gesture tracking still require cameras and light sensors. This is also increasingly true of machines that need to understand the world not just to augment our realities, but for their own reference.
“Positional tracking and placement of virtual objects are two of the biggest challenges in XR faced by developers,” said David Chen. “From a hardware standpoint, accurate scanning technology for digital duplication, networks that render physical environments in real-time, and display technologies are necessary advancements needed to make the metaverse a reality.”
Chen is the co-founder of Orbbec 3D, a company that makes 3D cameras and depth sensors. These sensors are used for body tracking, 3D assets, object and facial recognition, 3D vision for smart devices, and more.
“The devices and systems that enable this new era are well underway and, within a few years, will allow the metaverse to flourish,” said Chen. ”Many of the breakthroughs to come will be as a result of 3D scanning.”
While a number of these applications involve objects and assets, Chen reported that the company is increasingly putting humans on the end of the lens – specifically through virtual production and remote collaboration.
“We are currently working on immersive meeting and live broadcast projects with partners to reduce the visual glitches in these applications caused by conventional 2D technology,” said Chen. “The pandemic showed the value working from home provides. At Orbbec we are trying to make people feel more natural and comfortable in their virtual spaces.”
Maintaining Distance
A lot of the time when we talk about XR solutions, we think about reducing distance and bringing the world together. There’s a lot to be said for that, but the gift of AR, in particular, is that it also draws on the people and places that are physically around us. These positioning and tracking companies contribute to that universe through their revolutionary approaches.
