How Nex Is Flipping AR Games, and Why That’s a Great Thing

Augmented reality has a lot of promise for social and active gaming applications. An AR game’s use of the individual and their actual surroundings invites a connection to others and to physical space itself that tends to be absent from other kinds of gaming – including VR gaming. However, XR games are typically either social or active. Nex thinks that games should be both.

Meet Nex

Nex is a hardware and software developer making “motion games.” That is AR games that use motion as the only input. This isn’t entirely new. For example, once the level is started, games like Beat Saber only register motion – that motion is tracked with a controller, but the controller doesn’t provide other forms of input.

“Our games only require a camera and a device with sufficient processing power,” Nex CEO and co-founder David Lee said in an interview with ARPost. “Today, that processing power is reaching living room entertainment devices.”

That includes connecting compatible televisions to a mobile phone or another connected camera and compute box, but it also increasingly includes televisions with their own built-in cameras. Nex software can recognize multiple people with a single camera for AR games played together and on the same screen.

The two main offerings from Nex are a hardware camera and compute box currently in pre-production, and games created by the company’s four internal game studios and six outside partners using the “Motion Development Kit.”

Is it XR?

Something about Nex feels like it can’t be XR. That’s possibly because there’s no near-to-eye display. There’s no head-worn device – there’s not even an arm’s-length screen. However, if we think about the way that we’ve always defined XR, those aren’t things that we insist on.

We say that AR is virtual elements overlaid over a live view of the physical world. We often think of viewing that through a lens as with head-mounted AR, or through a camera as with mobile-based AR. Nex admittedly flips that standard model – but it still fits the bill. And it has its advantages over “conventional AR.”

“We flip it around so the phone sees you […] and leveraging the biggest screen that most people have,” said Lee. “You can have the effect of a bigger screen by mounting it on your head but that’s not a communal experience.”

Those who have been around the tech world for a few 24 hours may recognize this approach. Over ten years ago, PlayStation Move used a similar model, as did Xbox Kinect. If the camera-flipped AR game is the future, why is the past littered with these experiences? In part because AR isn’t the only tech involved. Nex also relies on artificial intelligence that wasn’t around in 2010.

“At the time, there was no AI, so they had to have a more complicated camera system,” said Lee. “What was missing from those previous generations of games was the NPU – the neural processing unit.”

Those games were fun – and ground-breaking at the time – but their reliance on a console limited their success and led to unsustainable upkeep burdens on the companies. Neither of those constraints is true of Nex.

A Look at Nex Games

I haven’t yet had the opportunity to play Nex games myself. I did get to watch Lee and one of his colleagues playing some of the games on a live video call.

Party Fowl is a collection of party mini-games that looks similar to JackBox. The package will be available as an annual subscription and includes a mix of AR games and what Lee called “VR-like experiences.”

In one AR game, rotating your hips flies a helicopter. In another game, players represented on screen as a chicken squat to lay eggs and fill a basket.

Another game, Air Racer, is a “flight simulator” in which players pilot an airplane through an obstacle course by moving their hands. Controls include direction, speed, and elevation.

While Nex is focused on games at the moment, I might be more interested in a fitness application from the company. Lee doesn’t see them as separate experiences.

“Movement is a natural way to play. As human beings, we’ve been playing for a very long time, and most of our games involve movement,” said Lee. “These games invite you to move more and also deliver those benefits in a gamified way.”

One experience really spoke to me as a potential showcase of a whole genre of experiences. The game was an episode of the children’s show Peppa Pig, in which gamers chose characters from the show and engaged in their favorite activity – jumping up and down in muddy puddles. The game was created with partner Hasbro.

“It’s not just watching – the family can be invited to join in the fun as well,” said Lee, who described the experience as “productive, independent playtime for the kids.”

Lee further described “the highlight of his career” as when his daughter got his mother into Nex games so that they could play together.

Experiencing Nex AR Games

I hope to get the opportunity to try out Nex AR games, and it sounds like I’ll get the opportunity soon enough – one way or another.

Nex AR games including Party Fowl and Sky Racers are already shipping as pre-installed apps on the Sky Live interactive camera. In fact, most of the motion games available on the camera are by Nex. For Apple users, Nex also works with the Continuity Camera feature.

Nex Playground – a camera box for Nex games compatible with most modern smart TVs – is currently in pre-order with the first orders scheduled to ship before this year’s holiday season. But, one day, external devices won’t be necessary at all as televisions ship with cameras and more computing power onboard.

“TVs don’t have really good processing yet. The memory is still quite limited but this is the beginning of these use cases,” said Lee. “This will be in a lot of living rooms and it begins with Nex pioneering this technology and showing the world what is possible.”

“The iPhone Moment for TV”

From AR games, to fitness applications, to just using hand gestures to navigate traditional media, Lee and Nex have an exciting vision for the future of television. The whole thing does feel like AI and XR reaching back into history to pull some of entertainment’s near-misses into the future where they belong.