Book Review: Mark Pesce’s “Augmented Reality: Unboxing Tech’s Next Big Thing”

Mark Pesce’s book Augmented Reality: Unboxing Tech’s Next Big Thing comes to shelves in January, but Polity Books put an advance copy in our hands at ARPost.

The book takes a long look at augmented reality’s past in an attempt to warn us about its potential future.

Pesce is an award-winning columnist published in The Register and IEEE Spectrum. He also hosts the award-winning podcast The Next Billion Seconds.

Augmented Reality Changes Our Relationship to Space

The very first page of Pesce’s book puts the reader in July 12, 2016. Setting up augmented reality giant Niantic as one of the book’s principal developers and villains, Pesce describes “A Riot in Rhodes” referring to crowds of Pokémon Go users descending on Peg Paterson Park in Rhodes, New South Wales.

The story doesn’t only introduce AR and Niantic, it introduces what may be one of the most significant recurring themes of Pesce’s book: “Pokémon Go changed players’ relationship to space, and changed their behavior within it.”

Throughout the rest of the work, Facebook and Niantic are established as strong examples of how AR can go wrong and how it may continue to go wrong in terms of user-data manipulation for profit.

The (Long) History of Augmented Reality

From the introduction, Pesce moves onto a history of computing and user interface beginning – where else – with Classical-Age Egypt.

I wasn’t a close student of computing technologies before I started into tech journalism a couple of years ago. So, for me, Pesce’s history of technology – which takes a couple of pages to get around to names like Wiener, Licklider, and Sutherland – was charming and informative. If you didn’t know those names you should buy the book just for chapter one.

However, if you want less of a big-picture approach, this history of AR that begins two thousand years ago might seem a little lost in the weeds.

Almost twenty pages into this history, Pesce introduces how “three key technologies removed the roadblocks to realizable augmented reality.” Pesce identifies these technologies as Microsoft Kinect, Keyhole (a forerunner of Google Maps), and the smartphone by way of Google Cardboard. As Pesce writes near the close of chapter one:

“When the three key technologies of depth mapping, geospatial datasets, and inexpensive but powerful hardware came together in the second half of the 2010s, it became possible to design fully-featured augmented reality tools.” 

AR Hardware as Surveillance

From the book’s second chapter to its last, the book becomes less of a history and more of a social commentary built around milestone events in AR. Pesce introduces Microsoft’s original HoloLens as a “once in a decade, world-changing product” on par with the Macintosh computer and the iPhone. From there, he posits that all AR hardware is surveillance hardware:

“When hundreds of millions of people don AR mirrorshades, each of them actively sensing, analysing, and integrating data from the world around them, we will unintentionally co-create a world where we have each placed one another under the tightest surveillance in human history.”

The smartphone is introduced as the beginning of a global addiction to screens, and Google Glass and smart glasses are introduced as the next big fix:

“Believing that they’ll recover their agency from the hegemony of the smartphone screen, millions of users will buy these spectacles, achieving the desired release as they look up and out, rather than down into the device. But at what cost?”

Facebook’s involvement in AR is introduced as an example of self-interested companies using our data against us:

“AR devices need to read us in order to create their illusions. We become an open book. Once they read us, they use what they’ve read to write upon us. Who gets to read those books?”

Much of one of the chapters is largely an annotated F8 speech that reads like an imagined dialogue between the author and Mark Zuckerberg. Snapchat is introduced as an example of how the unrestrained ability to write on the environment puts everyone at risk of “bad actors.”

Writing the World

One of the keystones of Pesce’s book is that augmenting reality gives the sense that the world is speaking for itself. We will continue to interact with the world but the world will also have the ability to interact with us.

Pesce’s vision of augmented reality as a tool that promises the best but may deliver the worst is expressed in terms that are sometimes hopeful, sometimes terrifying – profound, but often very dense. He concludes a chapter called “The Web Wide World” with the less-than-crystal sentiment:

“Where the whole world has become the Web, and the Web the whole world, and the world seems to speak for itself, we must always ask ‘Who speaks for the world?’”

Towards the end of the book, Pesce does recommend some solutions to some of these problems. On the very last page, Pesce writes:

“The path we walk at present represents the consequences of all the decisions we have already made. We can learn from that past, and perhaps avoid its pitfalls.”

If Pesce’s outlook seems dark, it may help to notice one thing: for as many “bad actors” as there are in augmented reality, there are also many good actors.

Darabase works to help property managers understand and work with AR creators in the intersection of digital and physical property. The Open AR Cloud Association works to create a world in which organizations cooperate rather than viscously compete. The Open Geospatial Consortium helps to ensure that those creating digital maps of the physical world are working with the same maps and that those maps are created fairly. The XR Safety Initiative is working so that user accessibility and human respect is built into the experiences that they access. None of these organizations receive so much as a passing mention in the book.

A Book of Much Brain and Little Heart

In the end, Pesce’s book drafts an expansive and holistic picture of how augmented reality works as a system of machines. In doing so, it leaves out much of the warmth and humanity that comes from the people creating that system.

I wouldn’t try to argue with any of Pesce’s positions regarding the care that we should take with augmented reality. But I would argue that much of that care is already being taken and, in some cases, woefully overlooked.