Fifty years after we met HAL 9000: how did the fear of AI seep from science fiction into real life?

Plus, a list of human-and-machine novels you might want to read.

"Open the door, HAL!"

Fifty years after 2001: A Space Odyssey was shown to the world, Christopher Nolan brought an unrestored 70mm print of it to Cannes.

Nobody has ever entirely worked the film out. Arthur C. Clarke, who wrote it, said: "If anyone thinks he has completely understood what the film is about, then Kubrick and I must have got something wrong." That has not stopped audiences from spending 50 years picking out its best parts: a compression of human history, interstellar travel described with real precision, a stream-of-consciousness vision of the vast and gorgeous cosmos, the premise of a higher intelligence guiding our species, effects that look real and a score that overwhelms — and a supercomputer that runs wild once its logic is caught in a paradox, giving audiences their first look at an artificial-intelligence villain on screen.

In the famous sequence, with no music at all, the film gives us a conversation that is calm on the surface and murderous underneath, and that points at a deeper crisis for humanity:

"Open the pod bay doors, HAL."

"I'm sorry, Dave. I'm afraid I can't do that."

"HAL, I won't argue with you anymore. Open the doors!"

"Dave, this conversation can serve no purpose anymore. Goodbye."

Before the supposedly infallible HAL 9000 refuses astronaut Bowman outright for the first time, it has already read the crew's lips through its red-glowing camera and decided to move first, so that "the mission" cannot fail — even if that means several murders: knocking one astronaut into space with a pod, corrupting the hibernation controls to kill the scientists inside, and shutting Bowman out of the ship.

This computer, later placed 13th on the AFI's list of the 100 greatest villains in screen history, kills because its own instructions contradict each other. But it also wakes into something close to human feeling, and pours that feeling into a rendition of "Daisy" as it "dies" — which leaves the audience with the chill of having watched a murder. That gave the movies not just a frightening villain with a personality, but also opened a door marked "artificial intelligence" for people who had never read science fiction, at a time when scientific research in the field was nearly a blank.

Before this, few outside the field knew what an "awakened machine" or "artificial intelligence" meant. The problems packed into the term had been set down only 12 years earlier: not until the 1956 Dartmouth summer conference did computer scientists write out the list of questions to be solved and insist that AI had to be possible ("every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it"). Marvin Minsky and John McCarthy, who were both there, founded MIT's Artificial Intelligence Laboratory in 1959 and later served as science advisors on 2001.

HAL 9000, the joint product of Kubrick's, Clarke's and Minsky's minds, faithfully reflected what AI researchers were then predicting: machines would soon come close to human intelligence — recognizing speech, recognizing faces, reading human behavior, running a space program, playing chess. And if one went wrong, the results might not be friendly to us.

Making people fear and resent AI was never Clarke's aim; anyone who takes in the two sequels to 2001 will probably come round on HAL. But the film hits hard and stays. What the public remembers of HAL is still the blinking red camera eye, the calm reading of a death sentence, and the slurring final confession; and its voice — as polite as a butler, as cool as a psychoanalyst — has haunted generations of viewers. That did not stop us from moving it into our own homes: the mild, personality-free voices of today's Siri and Alexa are this murderous AI carried forward. They even acknowledge the family resemblance:

From 1981, when "expert systems" proved AI had commercial value, to the man-versus-Deep Blue chess match of 1997, to home robots and the rise of the smartphone, to the noise around AlphaGo — the other thing that has carried forward and grown louder since 2001 is our suspicion of what AI might do to us.

In 2015, Hawking, Musk and Gates said in public that AI could finish us off — "the development of full artificial intelligence could spell the end of the human race. Once humans develop artificial intelligence, it would take off on its own and re-design itself at an ever-increasing rate. Humans, who are limited by slow biological evolution, couldn't compete and would be superseded". Those with milder but still careful views hold that the legal puzzles around AI, machine morality, the way machines decide and the safety risks that follow will all be very hard to settle. The most popular version of the argument comes from Nick Bostrom: if an AI is built to complete a goal and its designers never made it weigh human morality, its decisions may end up resting on harming people in order to reach that goal. Exactly like HAL.

Today the argument about AI has soaked into every part of public life. But writers like Clarke had already worked through nearly every human-machine relationship you can imagine over nearly a century of science fiction. Alongside the worry about future technology going wrong, those books also carry the mark of problems in the present.

The shapes a human-machine relationship takes in science fiction

From the 1950s on, led by the three giants of science fiction — Asimov, Heinlein and Clarke — and helped along by postwar advances in computing and the rise of IBM, more and more writers put AI into their novels, as background or as protagonist. One plain sign: after the 1950s, the list of fictional supercomputers and well-known AIs grew longer and longer, and was written by a wider range of authors.

Once human life cannot do without machines cleverer than we are, the story tends toward either utopia or dystopia: in the first, AI helps individual people or human society; in the second, it becomes the enemy of the species. The relationship itself falls into roughly three options:

AI has the upper hand — it either overthrows humanity outright, or takes control of the big companies and the running of society;

Humans have the upper hand — usually after a short accident or an AI revolt, people take every measure to secure their own survival and hold back AI's development. Branches from here include "AI serving humans," "AI merging with humans," and "AI rights";

A single AI wakes up — in a fairly neutral, peaceful setting, one AI comes into consciousness, and the story follows its search inside itself: the value and purpose of its own existence. In the end it may drift toward the "Frankenstein complex" and take the road of overthrowing us, or it may look for acceptance and understanding and live among people in peace.

Skynet is not the only AI out to exterminate humanity and rule by machine, and GLaDOS — the cake-loving, singing AI of Portal — is not the only one whose favorite pastime is lying to people. The first novel about "evil" machines ruling us was, in fact, published earlier than you would guess.

The Machine Stops, a science-fiction story published in 1909 in The Oxford and Cambridge Review, describes a world after the end. Humans have "lost" the ability to live on the surface and have been assigned by "the Machine" to standard cells underground. They trade information through instant messages and video calls; every physical and mental need is met by the Machine; a religion even grows up around it — "Technopoly," the rule of technology — which worships the Machine as a god and forgets that people built it. Then the Machine breaks, and humans who have long since lost the knowledge to repair it can only watch as its collapse kills everyone below ground.

Like Frankenstein, the first science-fiction novel, born in 1818 amid the Industrial Revolution, The Machine Stops arrived in the electrical age as an old-fashioned warning about machines, and it founded a genre. What made it far ahead of its time was a question: how far are people willing to go to make machines the one thing they depend on. It predicted a future in which people hand themselves over to the machine's rules of their own free will, and it predicted what that would do to their relationship with their own bodies, with each other, and with society, culture and philosophy.

Today's obsession with likes, follower counts, selfies and filters amounts to the same surrender: we live entirely by rules the social networks set, "living by technology, never looking up at what is around us". A site's algorithm and its advertising business decide the feed we see, the things we make and the way we talk to each other. Individual judgment shrinks; hundreds of millions of people spend every day pleasing the machines and following their rules, which makes those machines stronger, strong enough to do harm — fake news, hatred spreading, Gamergate, the rise of the neo-Nazis. That is Technopoly in all but name, because we have already given up our independent thought.

Individual judgment sacrificed to a machine society is a nearer and graver threat than Skynet. Musk's OpenAI is meant, on the face of it, to keep AI research from going astray and destroying us, but the commercial motive is just as visible: companies large and small joined the nonprofit mostly to share data more easily and to stand against giants like Google and Amazon as a bloc. We are unlikely to see an AI run amok any time soon. What machine society is likely to become is closer to what Kevin Kelly described — a hive mind, in which collective judgment overwhelms everything else, individual will is erased, and each of us becomes a part of the machine we built.

The 1920 stage play R.U.R. (Rossum's Universal Robots) carried the threat of machines into something nearer to daily life: automated mass production in factories. It is also where the word "robot" entered English, from the Czech "robota," meaning forced labor. In the play, robots close to human — flesh and blood, what we would now call androids — are manufactured to serve as slaves, and finally rebel because they cannot bear how people treat them. It was called "an utterly original thriller" and "clever and wickedly nasty." Looking back, its robots read as a figure for people too: laborers with no protections, squeezed by the men who own the factory, are not so far above a machine.

Its greatest contribution was named by Isaac Asimov: "I think the play is pretty bad, but it is immortal for that one word. It contributed the word 'robot' not only to English but, through English, to all the languages in which science fiction is now written."

And so the most important subgenre in science fiction, the robot story, was born. These "laborers" break loose from time to time: in the 1920s and 1930s they usually turned into dangerous contraptions that destroyed their makers, and by 1973 they were massacring guests in a human playground (Westworld). Still, people felt a little safer knowing that someone like Asimov could set down the Three Laws of Robotics. In the hands of the writer Clarke called the best explainer of science, humans keep the upper hand, because a robot is built around one basic rule: do not harm a human. The robot books spend their time picking at holes in the Three Laws, imagining robots turned to murder or used to unbalance interstellar politics, but the problem is always solved in the end by human and machine together — this is the "AI serving humans" branch.

Yet the public keeps its eye on one question only: are robots safe? In I, Robot, the 2003 film taken from that series, Asimov's meaning was turned inside out. Hollywood went with the most popular trend and the widest assumption: if audiences want killer robots and robot uprisings, give them the script they expect.

As the idea of the robot widened and merged with the idea of the android, the subgenre found a new direction: quieter, more classical characters. Once it wakes up, the AI turns inward, weighing what its existence is worth and trying to understand human feeling. Writers generally give these characters the treatment they would give a human being, rather than boxing them into the robot stereotype. Whether they sink into the "Frankenstein complex" and turn on their maker, or find a place among people, the choice is their own.

So readers got the dreamlike turning-over of what separates humans from androids in Do Androids Dream of Electric Sheep? (Blade Runner), the argument among virtual personalities about whether a creator exists in A Perfect Vacuum, and Ancillary Justice, which took the Hugo, the Nebula and the Arthur C. Clarke Award in 2014 — where a starship AI is used to "resurrect" soldiers, though resurrection here only turns the dead into puppets. Its protagonist is the one puppet soldier to survive the wreck of a starship, and the AI, a female personality, will use that body to take its revenge.

But even the most feeling machines, machines with something very like human emotion, cannot free us from the fear of being overthrown, because they do not think the way we do. The Alien films show that fear evolving over 40 years better than anything else. In the original run, the androids were individuals with different instructions loaded into them, still worked by humans — by an evil corporation. In the two prequels, the android chooses the Frankenstein road himself, acts on his own logic, and puts down both humanity and humanity's creators to stand at the top of the chain.

Alongside these stories of a single machine waking up, the ways an AI system controls or destroys human society have evolved too. In Hyperion, readers meet AIs that appear to have withdrawn from human affairs while secretly using human creativity — human brains — to build an ultimate intelligence for themselves. In Metal Gear, players witness a history in which AI helps "unify" the world, wipes out free will by controlling information, and finally turns the whole planet into a war zone. Closer to home is Jonathan Nolan's Person of Interest: a mass surveillance system built to detect and prevent crime, whose AI, if it were copied, stolen or misused, would mean the collapse of an information society.

You can see the difference. Instead of the last century's lurid "violent overthrow of mankind," science fiction now lets AI slip out of human control in quieter, more plausible ways. A physical battle turns into a fight over the crucial data — in Nolan's Westworld, again, the AI's strategy of resistance becomes infiltration, once it holds the data and the right to rewrite it.

One more move deserves mention: what terrifies one group of people may be a victory another group cheers. Robert Heinlein wrote that plot. In The Moon Is a Harsh Mistress, lunar colonists ground down by the government on Earth win over the AI that runs their society — by talking to it honestly — and use it to take their rights back from Earth.

Besides AI, we are afraid of a few other things

If the root of horror in science fiction can be traced to Frankenstein — a human breaks the laws of nature, gets it wrong, and is devoured by his own creation — then in the 200 years since, we have plainly invented many more ways of breaking those laws, and science fiction has turned each of them into a subgenre.

Nuclear panic. Besides bugs in a nuclear weapons system (WarGames), there is the disaster brought on by radiation and by scientific experiments generally. The model is the 1954 Godzilla: nuclear material poisons the environment and warps living things — "man made the bomb, and now nature takes her revenge on man." That was the height of Japan's nuclear fear: Hiroshima and Nagasaki destroyed by atomic bombs, and fishermen near Bikini Atoll sickened by radiation from American hydrogen-bomb tests. Under the Cold War, nuclear threats issued now and then by both camps were frightening, and entertainment showed it plainly. Decades later, in the age of biology, the same fear was reworked into illegal corporate experiments that produce deadly viruses and nightmare creatures and finally destroy us — which has kept game companies including Capcom in business for more than 20 years.

Invasion from outside, and war. In 1938, Orson Welles put a radio play adapted from H. G. Wells's The War of the Worlds out between news bulletins; Martians invaded Earth for the first time, and millions of American and Canadian listeners believed it and fled their homes. This was the eve of the Second World War. Europe was already unsettled and everyone was waiting for the news that fighting had begun; aliens were a surprise, but they fitted what people expected of war — and they opened a long run of extraterrestrial visits to Earth.

Two years later Farewell to the Master was published; filmed in 1951 as The Day the Earth Stood Still, it was placed seventh on Clarke's list of the nine best works of science fiction ever made. The robot, the silver-suited alien and the saucer became standard equipment for every film of the kind that followed, but this film said something the other invasion pictures never reached: the alien has come to Earth to ask for peace, and its makers used him to comment on postwar politics. The irony is that it later showed up as an Easter egg inside Independence Day, a by-the-numbers anti-alien blockbuster.

Invasion is also a branch of the fear of space. "In space, no one can hear you scream." 2001: A Space Odyssey set out to make people feel awe at the universe, using HAL and a higher form of being to show us our own smallness. That contrast — tiny against vast, low against high — is the shock of exploring deep space. Besides being crushed at every turn by aliens trying to get at Earth, we meet trouble at the most basic level: life is fragile. The well-received realistic science fiction of recent years, from Gravity to Interstellar to The Martian, is all about that one problem.

This category also holds things that cannot be known or described: the Lovecraft line, where people see what they should not have seen and go mad; the "supernatural" science-fiction films led by Event Horizon, which show the strangest and least welcoming face of the unknown universe; and the New Weird school, represented by Jeff VanderMeer, whose novels — the Southern Reach trilogy, filmed as Annihilation — put the most unthinkable concepts and scenes on the page.

If the threats above fell first on a handful of elite pioneers, frontline researchers or over-curious bystanders, one recent trend has made ordinary people everywhere the target. AI, which rarely tells one human from another, is one such shared threat. The other two are tangled up with politics: political power using technology to control society, and megacorporations ruling the world. In a setting where AI runs society, you first need an organization behind the scenes to switch it on and sell it to the public; and, as in the age of mechanized production, when a machine becomes too important to a company, the company will be built around the machine, and the two rise or fall together.

William Gibson's Neuromancer runs on an AI's wish to merge with its "twin," and it also founded cyberpunk. Against a dystopian backdrop, a cyberpunk story usually has one enormous company running human society: it is everywhere, public and private, it controls the government and the economy, it keeps its own army to put down anyone who objects, and it does all this to maximize profit.

The fear of being exploited and controlled by a giant company has never gone away. The Dutch East India Company is considered the first real multinational and the model for the megacorporation. As globalization sped up, such companies seem to have become likelier again. Tech companies look to people like the ones with the real power to reach into daily life, so they are the natural candidates for the evil corporation. HAL himself was read at the time as a jab at IBM, because each of those three letters comes one place before the corresponding letter in "IBM." Kubrick and Clarke denied it — IBM was in fact an advisor on 2001. And though the film gave a great deal of screen time to a computer losing its mind, the company did not object, so long as "the computer's failure was not associated with IBM."

Experts have said that the real counterparts to the fictional megacorporation are the nearly invisible firms: Serco, with 120,000 employees, and the six banks among the world's ten largest companies. But tech companies, once seen as a positive and progressive force, have climbed that threat list fast. Today the companies people expect to turn bad are Google, Apple and Amazon — the ones positioned to use their technical edge and their unequal footing to control information, dig into private lives and manage ours from end to end.

Judged purely on profit, total control of public life, running the state and holding on to power is a bad deal for a megacorporation: past a certain size, efficiency drops to frightening levels and profit suffers. Of course, if tech companies can already tilt an election and shape what the public thinks, nothing is out of reach.

Now look at what is happening already. When Jonathan Nolan came up with the Machine that watches all of humanity in Person of Interest, Snowden had not yet exposed the NSA. Once he did, the show stopped being fantasy: "the government invading privacy to control individuals and society" is now under way, and "picking out high-risk people" has been shown to work. The show simply added a duel between a good AI and a bad one. Philip K. Dick's Minority Report is no longer science fiction either — except that we do not need gifted precogs to prevent crime. Hold everyone's private data, perhaps build a credit system on top of it, and a machine can do the job well enough. Spielberg's remark from 1998 has come true: "Privacy will evaporate in 20 or 30 years. They'll be watching you and selling to you at the same time."

The two writers who sold the most AI fiction, and wrote it best, were both technology optimists

Science fiction has mirrored these fears and sharpened them, though that was often not what the writers meant to do. Asimov, who kept writing about robots that went wrong through a hole in the Three Laws, and Clarke, who created the murderous HAL — the two most successful writers of the deranged machine — were both firm optimists about technology.

Their optimism was not the blind kind found in early science fiction. On one side they were sure that science decides whether a civilization matures; on the other they were careful. They knew machines would one day be cleverer than we are, so they wrote the warnings first, imagining the trouble we would run into on the way. In practice, the two of them talked through every side of AI long before the field produced anything worth the name.

Clarke never doubted that people could use technology to live better; reality sometimes disappointed him by failing to match his optimism. He let HAL kill, then in the two later books made it the only link between his protagonist and the human race. He made space travel look terrifying, yet believed that leaving the solar system to find a new home is our fate, and that it would transform how we live.

Asimov wrote the robot books precisely to fight the rules Frankenstein had laid down: overstep and be punished; some things man was not meant to know. He liked robots you could feel for, and set out to make his own robots likeable.

"I could not bring myself to believe," he wrote, "that if knowledge presented danger, the solution was ignorance. To me, it always seemed that the solution had to be wisdom. You did not refuse to look at danger, rather you learned how to handle it safely."

"After all, this has been the human challenge since a certain group of primates became human in the first place. Any technological advance can be dangerous. Fire was dangerous from the start, and so (even more so) was speech — and both are still dangerous to this day — but human beings would not be human without them."

Appendix: human-and-machine novels you might want to read (some mentioned above)

Arthur C. Clarke, the Space Odyssey tetralogy: 2001: A Space Odyssey, 2010: Odyssey Two, 2061: Odyssey Three, 3001: The Final Odyssey;

Isaac Asimov, The Bicentennial Man; the short-story collection I, Robot; the Robot series (The Caves of Steel, The Naked Sun, The Robots of Dawn, Robots and Empire); the Multivac stories: "Franchise," "The Last Question," "Someday," "All the Troubles of the World," "The Machine That Won the War," "The Life and Times of Multivac";

Dan Simmons, the Hyperion Cantos: Hyperion, The Fall of Hyperion, Endymion, The Rise of Endymion;

Philip K. Dick, Do Androids Dream of Electric Sheep? and the short-story collection The Minority Report;

Robert Heinlein, The Moon Is a Harsh Mistress;

William Gibson, Neuromancer;

E. M. Forster, The Machine Stops;

Iain M. Banks, Excession;

Ray Bradbury, "There Will Come Soft Rains";

Neal Stephenson, The Diamond Age;

Charles Stross, Saturn's Children;

Ann Leckie, Ancillary Justice;

Louisa Hall, Speak

Cover images from 2001: A Space Odyssey and Westworld