At Google's Annual Developer Conference, Everything It Wanted to Talk About Was AI

From Google Assistant to Waymo's self-driving cars, almost every product Google discussed came back to artificial intelligence.

On the morning of May 8, local time, Google held its I/O 2018 developer conference at the Shoreline Amphitheatre, near its Mountain View headquarters.

CEO Sundar Pichai hosted the opening keynote, as he always does. The talk was built around Android: Google announced the first beta of Android P and updated Photos, News, Maps, the voice assistant and more.

There was no heavyweight hardware this year — only smart speakers with screens, made by JBL, Lenovo, LG and others, due in stores in June.

Near the end of the keynote, Waymo, Alphabet's self-driving car unit, made its first appearance in an I/O main presentation.

Every one of these products and features came back to the same thread: artificial intelligence.

Google's Assistant Sounds More Human — and Answers Like One

Pichai set up the feature news with a few technical demonstrations. He played a clip from a news commentary show in which two hosts argued heatedly, cutting each other off again and again. The upgraded Google AI can pull the two voices apart and write separate captions for each speaker, without being thrown by the interruptions.

After a second example — AI helping diagnose diabetic retinopathy in developing countries — Pichai turned to the upgrades to Google Assistant.

First, the Assistant will get six new voices. All six are synthesized with WaveNet, the technology Google announced last year, which works not word by word but by modeling the raw sound waves of speech, which makes the result far more lifelike. One of the six was trained with the help of the R&B singer John Legend, who recorded a series of words and full sentences at Google; the synthesized voice sounds a great deal like him.

The biggest functional change is what Google calls "continued conversation." The Assistant can follow a subject as it shifts from sentence to sentence, and it can handle several questions inside one sentence.

Until now, talking to the Assistant meant one instruction per sentence, and every sentence had to carry all the information — it listened one sentence at a time.

Now you can talk to it the way you would talk to a person: ask a team's score in one breath, set a reminder about where to pick someone up in the next, then return to the first thread with something like "when do they play next?"

The "multiple actions" update turns on the word "and." You can give the Assistant several tasks in one sentence — "what's the weather tomorrow and order me some food delivery" — and it will carry out both.

The most striking demo was the Assistant's ability to hold a phone conversation: while you are busy, it can call a barbershop or a restaurant and book the appointment for you.

In the demo, the barbershop employee was curt — what do you need? What time? Which day? — and the Assistant answered like a person: "Today at 10, I want a haircut." In the restaurant demo, the person on the line was clearly meant to be difficult, saying things like "we can't take a booking today, but you can just come in on Wednesday — so which day do you want? How many people?" The Assistant hesitated for a beat, then read the meaning correctly.

"What technology means to you depends on where in the world you are," Pichai said. "For someone like me, who grew up in a world without a phone, I remember clearly how technology changed lives. That is why we are committed to bringing digital technology to communities everywhere."

Google also announced new hardware built around the Assistant: smart speakers with screens. They look like digital photo frames with a speaker attached. Beyond what the Assistant already does, they can play YouTube Live on the screen, show the feed from a home security camera, or pull up a recipe with step-by-step instructions by voice.

The smart displays from JBL, Lenovo, LG and others go on sale in June; the Assistant's new features arrive over the next few weeks.

The Next Android Has AI in the System — and Worries About Your Screen Habit

Google spelled out what changes in Android P, in three directions: intelligence, ease of use and digital wellbeing.

The first, as everywhere else in the keynote, is AI. Android P uses it in several places, starting with battery life. The phone learns which apps you open most and decides when to shut the others down entirely. Google says this feature, Adaptive Battery, cuts CPU activity by 30%. Adaptive Brightness studies your habits in the same way and lights the screen less often and less brightly during the hours you rarely pick up the phone.

In Android P, AI becomes a system-level function that guesses what you are about to do. Type "Infinity" into the search box and a trending result for Avengers: Infinity War appears, with a shortcut to buy tickets at the bottom of the results. Type "Lyft" to open the ride-hailing app and a "ride home" shortcut appears below the search box; tap it and the pickup and drop-off points fill themselves in.

Ease of use mostly means a new interface. Google's universal search box has moved to the bottom of the screen, just above the home button. The three navigation buttons have been cut down to one, and the big actions are now gestures: swipe sideways on the home button to flick between recent apps, swipe up to open the full multitasking view. The result looks a little like the gestures on the iPhone X.

The other big change is digital wellbeing.

The feature Google calls Digital Wellbeing is, in plain terms, an anti-addiction mode. The system now includes a dashboard that counts how long you use the phone each day, which apps take the most time and when you use them most. You can then set limits on those apps — block a video app during working hours, for instance. Slack off at the office and tap its icon, and the system shows a grayed-out icon and a line of text telling you to stop.

A second feature, Wind Down, cares about whether you go to bed on time. You set a bedtime; after it passes, the phone not only enters do-not-disturb and stops showing notifications, it also drains the screen to black and white to nudge you toward sleep. The same feature is wired into automatic do-not-disturb — turn the phone face-down on a table and it switches on by itself.

The Android P public beta opens today. Eleven phones can install it with official support, among them the Google Pixel line, the Essential PH-1, the Sony Xperia XZ2, the Xiaomi Mi MIX 2S, the Nokia 7 Plus, the Oppo R15 Pro and the Vivo X21.

Waymo's Cars Turn Up at I/O, but Only to Say Hello

Waymo's appearance in this year's keynote was something of a surprise. In the December 2016 reorganization, Waymo was split off into Google's parent company, Alphabet, and is no longer a Google subsidiary.

Waymo CEO John Krafcik and CTO Dmitri Dolgov took the stage, their first time at Google I/O. They spent a few minutes on what AI does in self-driving research, arguing that it is essential to teaching a car to make the right decision on different roads and in different traffic.

"We're not just building a better car," Krafcik said. "We're building a better driver."

Deep learning, Waymo says, has made its pedestrian detection 100 times more accurate. In March this year, when an Uber self-driving car struck and killed a pedestrian, the car took the woman for a plastic bag and drove straight into her instead of braking.

"Our cars need to understand how every object interacts with every other object, and behave in a safe and predictable way," Dolgov said. In his demo, Waymo's car correctly predicted that a pedestrian would cross against the light, and slowed down before it happened.

People in traffic behave differently from one another; unlike lane markings, road signs and guardrails, they share no obvious features. Waymo is now teaching machines to understand human behavior, and that is a place where AI genuinely helps. The company is also trying to use it to fix radar's poor accuracy in snow.

Waymo has now driven about 10 million kilometers on public roads and logged more than 9 billion kilometers in simulation.

Its next big move will be a self-driving ride-hailing service in Phoenix, Arizona, later this year; the details are still unknown. A few months ago Waymo revealed that it was testing self-driving freight trucks for Google's data centers in Atlanta, and it announced a deal with Jaguar to build an all-electric self-driving fleet.

Maps, Mail, News, Photos: Every App Update Runs on AI

Gmail announced Smart Compose, which reads the context of an email and suggests the next few words. It extends Gmail's Smart Reply and reaches users in the coming weeks, provided they have switched to the new Gmail.

Smart Compose runs in the background: type "haven't seen you in a while" and it offers "hope you're doing well"; tap the suggestion and the text fills itself in. Google hopes this makes writing email faster and cuts down on grammar and spelling mistakes. Whether it works in Chinese is not yet clear.

Google Photos can color black-and-white pictures. Google Photos, which serves 5 billion views a day, keeps getting smarter: it can now add color to old black-and-white photographs automatically.

It also reads a photo's exposure and brightens it where needed, and spots which of the people in a picture are in your contacts, then asks whether you want to send it to them. All of this arrives in Google Photos over the next few months. For developers, Google released a new API that lets them build Google Photos into their own apps.

Google Lens reads text in a picture — and now lets you select it. The new Google Lens beta will be built into the stock camera app on ten devices, including the Google Pixel 2, the LG G7 ThinQ and phones from Xiaomi and HMD. Its image recognition can tell you more about the books, buildings and artworks it sees.

It works like this: you take a picture, and machine learning picks out the pixels that form text to extract more detail. If the picture is of a product, Lens will sometimes offer a shopping link. With a photo of plain text, Lens lets you select and copy the words right on the image, much as Google Translate does with pictures.

Google News gets a rebuild. "I think there is more journalism being produced today than ever before," Pichai said. "When people need it, they turn to Google, and we have a responsibility to deliver that information."

Google says it uses reinforcement learning to study reading habits, so that when you open the app the first five stories are all subjects you care about.

The new Google News also adds Newscast, a news briefing of its own that looks a little like Instagram Stories: a video, a quote or a summary that gives you the basics and lets you decide whether to read further.

For anyone who wants to go deeper on a subject, Google News is betting on Full Coverage. It gathers the different accounts of an event and the live discussion on social media, and lays them out in chronological order.

The one-click way to subscribe to a publication, previously available in the browser, has also moved into the app, with more than 60 major outlets on the list.

Google Maps adds AR navigation. Google calls it a "visual positioning system": navigation through the camera, based on image recognition. You point the phone at the street ahead or at a building, and the bottom half of the viewfinder shows Google Maps with the route, the estimated arrival time and the distance left, while the top half shows live directions.

Google Maps can also point out places it thinks you will like, based on your interests and on the places you have rated well before.

Two years after Google made AI its center of gravity, this year's I/O was where all of that surfaced at once. Set against its only rival on the market, iOS, Android appears to be walking in an entirely different direction. In the keynote Google stressed that Android is open, and that open choice produces innovation — which sounded like a dig at someone.

The pity is that, apart from the anti-addiction and ease-of-use features in Android P, almost nothing shown at this year's I/O will mean much to users in mainland China.

Cover image/Google