MR Company Nreal Talks Hand Tracking and New Partnerships

It’s impossible for anyone to keep an eye on everything happening in the extended reality space. However, Nreal is one XR company that I recommend keeping an eye on. A recent six-page press release detailed plenty of things to be excited about.

Catching Up With Nreal

To really appreciate all of Nreal’s announcements, it helps to understand where they are right now.

I’ve already said that they’re one of the most exciting new companies out there but not knowing about them is understandable. The mixed reality company doesn’t have a model on the shelves yet, but they are expected to be available very soon. Developers have had access to SDK since July.

The company makes one of the lightest weight models out there, despite also having impressive specs like a 52-degree field of view. They do require a hard-tether to a smartphone. The smartphone is also how most of the applications are controlled.

The glasses are also compatible with some third-party controllers. However, the first major announcement in their recent release has to do with controls.

Hand Tracking With Clay AIR

“By offering hand tracking with Clay AIR, Nreal offers its developers and users of the Nreal Light Consumer Kit an added degree of flexibility in how they might want to interact with their MR environment,” Nreal CEO and Founder Chi Xu said in the release. “Whether it’s hand tracking, their 5G smartphone as a controller, a third-part 6DoF controller, or even in the near future eye tracking in partnership with 7invensun.”

There’s already a lot there to unpack. Let’s start with some of those company namedrops.

Clay AIR is an XR company that specializes in commands using things like distance changes as computer inputs. That’s the kind of technology that allows hand tracking. This release is the first that we’ve heard of them working with Nreal but it makes sense. Clay AIR was already working with Qualcomm, who was already working with Nreal.

“Consumer-friendly hardware powered by seamless hand tracking and gesture recognition technology, is a necessary catalyst to make AR accessible and practical on a global scale,” Clay AIR CEO Thomas Amilien said in the release. “This partnership marks an inflection point with the first integration of hand tracking via embedded monochrome cameras already used for spatial tracking in AR.”

7invensun focuses on eye-tracking. While it may sound like a thing of the future, eye tracking has been around for a while. However, it’s mostly been used for things like usability testing. The idea of using it to control devices is fairly new. However, Nreal isn’t the only XR company promising this software. Varjo and Pico have it so far – and both of those are far more expensive than Nreal.

Partnership With DT

Consumer-friendly hardware is not the only thing necessary for global adoption. That’s partially because of the computing power required to run things like hand and eye-tracking.

Without getting too far into the tech of it, hand tracking works by mapping the hand and interpreting its movements. Eye-tracking is a little simpler but it still requires assigning meaning to the eye’s movements. Both of these require machine learning.

Machine learning requires a lot of computing power. Having that computing power provided by a mobile phone at a practical speed requires 5G internet. Not all internet providers carry 5G internet, so many of Nreal’s recent announcements have involved partnerships with those that do. So far the Chinese-based MR company is supported in China, Korea, and the United Kingdom.

That brings us to their next big announcement.

“At Deutsche Telekom we believe mixed reality will change the way people work, live and interact,” DT’s SVP of product, innovation, and customer service, Omar Tazi, said in the release. “By providing key technologies, from edge computing to the best connectivity, combined with the best content for entertainment, such as music concerts, sports events, or even gaming, Deutsche Telekom together with Nreal will unleash the full potential of mixed reality.”

Deutsche Telekom is the largest provider in Europe, particularly in central Europe. However, their subsidiaries (read “T-Mobile”) are active on virtually every continent. That includes North America.

Spatial Support

The final big announcement from the release involved support for a new app.

Spatial is an AR company specializing in collaboration. While a number of companies offer this resource, Spatial is unique in that it offers lifelike avatars. A number of other features, including persistent rooms and 3D modeling, make spatial unlike any other service.

The partnership is important because of Spatial itself but also in principle. Nreal has proven itself when it comes to casual uses including gaming. However, with huge corporate interest in XR, supporting productivity apps like Spatial helps to position Nreal as more than just a consumer product. The MR company can have a considerably larger market by offering industry tools as well.

A Competitor out of the Gate

Announcements like this one help to put Nreal in front of the game. By continually adding services before their product even goes to market, this MR company is doing more than generating a buzz. They’re setting themselves to come out of the gate as a serious contender to established companies.