How Virtual Does Reality Need to Be?

Enthusiasm for virtual reality has been rekindled, yet this round of results is no closer to everyday life than the last. What exactly do we want?

Regular article

On the day Microsoft unveiled HoloLens, my Samsung virtual reality headset arrived: the Gear VR Innovator Edition. "Glasses" is just shorthand; the more accurate name is "head-mounted display." To an onlooker, the Gear VR looks like a bulky pair of welder's goggles. From some angles it is nearly half the size of a human head, and its plastic build feels a little cheap. Next to HoloLens, which is no bigger than a pair of ski goggles, it looks like Motorola's original "water bottle" brick phone meeting a Nokia N95. A piece of cutting-edge tech went out of date in an instant.

But I was not disappointed. The strange thing is that the new era of virtual reality, which has still not arrived, already went out of date once, back in the late 1990s. In the first half of that decade, enthusiasm for virtual reality matched or even exceeded today's. Game makers such as Sega and Nintendo rolled out their own head-mounted displays, and in 1992 some media predicted that "virtual reality devices ordinary consumers can afford will appear within two years." That same year brought The Lawnmower Man, one of the most famous virtual reality films ever made, and Aerosmith's "Amazing" music video featured virtual reality scenes too. Just as everyone talks about social networks today, back then everyone was afraid of missing out on virtual reality. But soon after Nintendo released its Virtual Boy headset, the bubble burst. It is a familiar tech story: innovation that moved too fast, chasing demand that did not exist, produced a short-lived "next big thing." From then on, virtual reality became a punchline.

Some say tech fashions come back around roughly every 15 years. Since 2014, virtual reality and its cousin, "augmented reality," have again become hot topics in tech. In September 2012, 20-year-old Palmer Luckey raised US$2.43 million on Kickstarter to launch the Oculus Rift virtual reality headset. In August 2013, Oculus VR recruited the legendary game programmer John Carmack as chief technology officer, and at the end of that year the company raised a US$75 million Series B from Andreessen Horowitz. The real watershed came last year, when Facebook announced in March that it would buy Oculus VR for US$2 billion. Consider that Facebook paid US$1 billion for Instagram in 2012, but Instagram was by then a polished product with a huge base of active users, whereas Oculus Rift has still not released a single virtual reality device for ordinary consumers. The tech industry's and the capital markets' enthusiasm for virtual reality had come back from the dead.

In early 2015 we saw HoloLens. Unlike the virtual reality headsets of the 1990s and today's Oculus Rift, HoloLens is an "augmented reality" device. Virtual reality tries to build you an environment made purely of bits (hence "virtual"); augmented reality "overlays" bits on the real world of atoms, using the two-dimensional world to "extend" and "enhance" the three-dimensional one. That is why Microsoft's launch event showed scenes such as playing Minecraft on a coffee table, or having someone talk you through fixing a water pipe over Skype. A simple rule of thumb: if you can still see the real world (including your own body) after putting on the glasses, it is most likely augmented reality; if not, it is virtual reality.

Though both are still in development, augmented reality has a big advantage when it comes to promotion. Virtual reality rests on the premise that a two-dimensional screen cannot create a truly immersive experience, so its real effect cannot be shown on the two-dimensional screen of a computer, phone or tablet either. And because a head-mounted screen is enclosed, only the one person wearing it can see what it does. Augmented reality is different. Put a person wearing the glasses in a particular space, overlay some holographic images, and you can easily produce a stunning effect. In other words, with today's technology, the experience of augmented reality spreads easily through the media (whether the real device can deliver that effect is another matter), while the only way to experience virtual reality is to go and buy a headset yourself.

The Gear VR Innovator Edition (hereafter Gear VR) was developed jointly by Samsung and Oculus VR. Samsung seems impatient to build an "ecosystem" of wearable devices. In current American TV ads, the Gear VR appears alongside Samsung's smartwatch, the Samsung Gear, as a new generation of devices to complement phones and tablets, tempting gadget lovers who want to be a step ahead. But both Oculus VR and Samsung know very well that the Gear VR is not yet a mature consumer product. When you buy it on Samsung's website, you even have to tick a box before paying that reads, "I understand that this product is intended for developers rather than ordinary consumers."

In practice, although the Gear VR is not yet something ordinary users would consider buying, it is nearly plug-and-play. Remove the translucent protective cover on the front of the headset and you will find a MicroUSB connector in a recess. Plug in a Galaxy Note 4 (currently the only phone the Gear VR supports) and a voice prompt tells you to update the phone's system. Take the phone off, follow the on-screen instructions step by step, plug it back into the MicroUSB slot, lock it onto the Gear VR with the clips on either side, and you are ready to enjoy virtual reality.

Getting the Gear VR onto your head the first time takes some effort, but once it is on, everything else is easy. On first use, the system plays a tutorial video that shows you how to move around the virtual world, and then you enter Oculus's software store, which will feel familiar to anyone who has used any app store. With a small four-way touchpad and a physical button on the right side of the headset, you control the image in front of you and can browse, download, install and launch the store's virtual reality games and tech demos.

Interestingly, although the Gear VR is undoubtedly a cutting-edge product, and already quite polished, putting it on for the first time did not give me that stunning sense of glimpsing the future. One reason may be that the ideas behind virtual reality have been described in the media too many times since the 1990s: 360-degree panoramic images, being transported into a scene while sitting motionless on a sofa. None of this is wrong, and words truly cannot match the experience of actually wearing the headset. Unfortunately, you can be almost certain that what you experience today is not what virtual reality will really look like. The Gear VR is too heavy and too big, the picture is not sharp enough, and the store has too little content. In an ideal world, time would solve all of these problems, but the real world has too many unpredictable factors: support from third-party content providers (in games and film), how users perceive the technology, the arrival of other new technologies, and so on.

This brings us back to HoloLens. Without a doubt, both the general public and the tech community are several notches more excited about HoloLens than about the Gear VR or Oculus. As noted above, unlike a virtual reality headset you cannot appreciate without wearing it, HoloLens looks impressive in the media with little effort. It uses holographic projection to overlay images on the real world, trying to merge the two-dimensional and three-dimensional worlds into one, and like a magician pulling a rabbit out of a hat, the effect is truly magical. If you hand a friend the Gear VR today, he may well ask, "Why can't this one let me see the things in front of me and the images in the glasses at the same time, like Microsoft's?" But every dazzling tech demo faces an unavoidable question: can the real product, used in the real world, live up to the demo? Here Microsoft's track record is not reassuring. Take the motion-sensing game controller Kinect, overseen by the same person and arguably HoloLens's predecessor. The first Kinect went on sale in November 2010, but Microsoft had released a demo video of it at the E3 game expo in mid-2009. Anyone who has played Kinect can see the gap between the real thing and the demo. Suppose HoloLens likewise goes on sale a year and a half after its unveiling (already a fairly bold assumption): by mid-2016, will we be able to buy HoloLens glasses that deliver what this event showed?

In fairness, Microsoft, never known for understanding what consumers want, put real care into imagining everyday uses in this demo video. But as Peter Molyneux, a former senior Microsoft executive, put it, a pioneering piece of hardware badly needs a convincing set of first-party software at launch:

Hardware makers, whether it's Microsoft or anyone else, often like to say, "We really want to get this product finished so we can see what wonderful things third-party developers will make." If Apple had said that when it launched the first iPhone, we wouldn't have the iPhone! However much you spend developing the hardware, you should spend at least as much developing real software... Nintendo released the Wii only after it had made Wii Sports, so people wouldn't be left with just a few tech demos to play.

This is the problem virtual reality and augmented reality share right now. Oculus Rift and Gear VR are not yet consumer products, and HoloLens is not even on sale. The biggest unknown is whether, by the time the hardware is ready, a killer app like Wii Sports will appear alongside it, along with the kind of consumer awareness summed up in "Oh, I know the Wii, that's the one where you can play tennis."

Although they are often discussed together, virtual reality and augmented reality are two opposite technological ideals. Virtual reality's appeal lies in defying reality by creating illusions; it is an escape from reality. Augmented reality hopes to use technology to enhance human abilities themselves, so that we can cope with reality better. In this sense, novels, films, plays, music and games are all virtual reality already, and every tool humans have invented can be seen as an augmented reality device in the broad sense. So rather than a new technological paradigm, virtual and augmented reality represent an evolution of tools: from unintelligent to intelligent, from general-purpose to special-purpose. The telescope, for example, is a typical "unintelligent" (no CPU, hence no computing power) special-purpose (its only use is to see distant things) augmented reality device, while the computer is a typical intelligent, general-purpose (it can be used for all kinds of things) augmented reality device. The various glasses that depend on computing devices are clearly intelligent, but pulled between special-purpose and general-purpose, they have yet to find their footing.

Special-purpose virtual reality devices have long been used in fields such as medicine and entertainment. Medical students practice operations on surgical simulators made up of 3D glasses and simulated instruments, and the cinemas at Disney theme parks around the world, which spray water, release scents and have things moving in the seats, offer a genuine virtual reality experience. These applications succeed because each is designed for a very specific, tightly controlled setting. When virtual reality is shrunk to the size of a pair of goggles and sold to consumers as an ordinary product, it runs into the tension between general-purpose and special-purpose: when consumers ask, "What can I use it for?", they want designers to give them concrete uses, but they also expect that "it should be able to do more than one thing." In a world where most people own at least one of a phone, a tablet and a computer, neither virtual reality nor augmented reality devices will be anyone's only computing device for the foreseeable future. Can we find a use that only virtual reality can fulfill?

Answering that question probably takes us into philosophy. You can say that a 360-degree panoramic film or game feels more immersive than one on a flat screen, but we can push further: how "virtual" does reality need to be before it is enough? In episode 20 of season four of the American sitcom The Big Bang Theory, Sheldon and Leonard play a motion-sensing archery game on the Wii. When Leonard refuses to mime pulling an arrow from a nonexistent quiver on his back, Sheldon says something telling:

Leonard, the people at Nintendo can only go so far in helping us recreate an actual athletic experience. We have to meet them halfway.

As I see it, "we have to meet them halfway" is not regret over the limits of today's technology but an affirmation of human agency. Every virtual environment humans have invented so far, that is, every work of art, asks the person experiencing it to "do some homework" to some degree; that is the half of the work Sheldon is talking about. Effort is not only a precondition for appreciating a work of art; it is itself a source of pleasure. (Remember the frustration of not understanding a film?) If one day we really could have a virtual orgasm without leaving the sofa (one of the most clichéd uses imagined for virtual reality), would that really be the wonderland we want as human beings?

I fear it would mean the end of humanity.

Cover image: PCadvisor