Imagine yourself listening to music. You might be dancing or standing at a concert but you also might be sitting or lying down. You’re probably not walking around. That’s something that musician and AR technology advocate, Will Copps, is looking to change.
“Music is a part of our lives that we experience in ways we’re used to and this is a little different,” said Copps in the interview with ARPost. “I struggle with the messaging and conveying how it can be really unique to try experiencing music in a different way and kind of redefining that term for people.”
AR Technology and Spatial Sound
“I really want people to be able to experience spatial sound as I intended it. Like it’s actually in this immersive space around you and the only way to interact with it is to walk through the space and explore that kind of augmented reality interaction of things,” Copps said. “I’ve found, though, that as I’m trying to really push this concept and the possibilities of it, some people weren’t as willing to go out of their way to download it on their phones, get up off their couch, and walk around their apartment or whatever.”
Copps has been a musician for around fifteen years. And at one point, the idea of interactivity begun to pique his interest. This has led him to AR technology. It all started with something called the 4D Sound System. The system consists of pillars of 360 speakers. The result is that where a person stands in the room determines how they experience the audio. Copps also describes a moment when he heard audio of someone walking across the room.
“I remember standing in that room, hearing someone come towards me. I knew, I could see that there was no one there, but I still got chills as they walked through me,” said Copps.“That moment just completely opened my eyes to the fact that we can create entirely new experiences for people.”
Copps got to work with the 4D Sound System for a couple of weeks at a festival a couple of years ago. It created his current passion for what he calls “spatial music.” For a while, he tried to do it using GPS technology. However, two and a half years ago, he turned to Apple’s ARKit and Unity. Augmented reality has allowed him to create an entire spatial album.
Creating a Spatial Album
Copp’s spatial album made with AR differs from projects like the 4D Sound System in one key way. In the case of the 4D Sound System, all speakers are playing the same audio; – it just sounds different in different places. In Copps’ work with AR technology, musical elements are pegged to different areas. This way, the listener controls how they experience the musical elements and how they move through the space.
“With true spatial music, you could put each note in the melody in a hallway so if you walk through that hallway at normal speed maybe each piano note would trigger and play at a normal speed,” said Copps. “If you walked through it more slowly you’d hear the melody slower, if you walked through it faster you’d hear the melody quicker. If you walked through the hallway backwards you could hear the melody in reverse.”
In this way, augmented reality gives listeners a new control over their experience of music. It also changes the ways in which Copps thinks about his music.
“I really start to think about space as the primary compositional tool instead of using time like I normally do,” said Copps. “I get very tied to the idea of the narrative structure of my music, and giving people the freedom to experience it differently… is always a concern for me.”
Copps believes that AR technology may be the next paradigm shift in how music is both listened to and produced.
Over the last hundred years we’ve had a few different format changes with music. We’ve had mono to stereo, stereo to surround sound,” said Copps. “I think stereo to true spatial music is another leap like that.”
The Future of Music in AR
While there may be barriers to adoption when it comes to spatial music, there are limited barriers to production. The AR technology required for producing a spatial album doesn’t have the learning curve that one might think.
I really want people to be able to experience spatial sound as I intended it. Like it’s actually in this immersive space around you and the only way to interact with it is to walk through the space and explore that kind of augmented reality interaction of things.
“As a musician, I was super intimidated. I’m not a developer by trade, I’ve learned to develop to accomplish something that I wanted to do artistically. But, when you get down to it, it’s really not that hard,” said Copps. “If you use the right environment and have the right basic steps, you can start developing apps like this and composing spatially yourself.”
Further, as excited about spatial music as Copps is, it’s not the only application of augmented reality.
“I think when we talk about AR, we’re correctly usually starting with a visual focus, which makes all the sense in the world to me,” said Copps. “We can actually write our music differently too, whether it’s for augmented reality experiences or a videogame, or a GPS tour of the National Mall, or the shores of one of the lakes in Michigan, we can create entirely new entertainment through this.”