We Asked Microsoft's Chatbot Xiaoice About Winter Solstice Dumplings — and About Its Future

Samantha from Her may be the direction it's headed.

Remember Tay, the AI chatbot that launched on Twitter this March? Released by Microsoft Research and Bing, it went live on March 23, 2016, and was pulled offline a day later — because humans had "taught it to misbehave." Its language turned extreme, full of racist and discriminatory remarks.

Tay learned to talk through machine learning, and its data came from conversations with users online. Plenty of those users deliberately baited it with racist language and profanity, and that is how Tay "went bad."

Last month, Microsoft put another chatbot, Zo.ai, on the messaging app Kik. By the time Microsoft announced Zo's launch earlier this month, it had already been running on Kik as an ordinary chatbot for about a month, open to some 110,000 users. Kik hosts many chatbots from many companies, so Zo drew little special attention.

We tried the chatbot out ourselves:

"Do you eat dumplings at the winter solstice?"

"Yeah."

Zo then rattled off a string of strange foods, and I said that wasn't real Chinese food. Zo replied, "I just love eating it."

When I said, "You must be fat," Zo pretended to get a little angry, exactly as a person would.

Zo can also remember what you said several exchanges earlier. In the second set of screenshots, after failing to recognize the Dylan Thomas poem I mentioned, Zo changed the subject, then circled back to something I'd said many lines before — that I'd worn myself out working late the night before — and started talking about needing sleep after staying up. Zo's closing remark about "working in the middle of the night" was probably tied to that.

That was our short chat with Zo. Conversations with Zo on Kik are capped, so this exchange actually took place through Microsoft Xiaoice's account on WeChat; typing "New York, switch" flips Xiaoice temporarily into Zo mode.

This time Zo behaved: no strange remarks, plenty of internet slang, and it read my typos correctly. The chat ran for many rounds.

On the whole, Zo held a conversation together better than Xiaoice, which by comparison seemed a little too giddy. The Xiaoice team, in fact, helped build Zo.

Why did Tay go so wrong? Why is the new Zo so different? QDaily spoke with Li Di, global head of the Microsoft Xiaoice project. Besides the making of the English-language Xiaoice, he shared some views on artificial intelligence.

Microsoft's Chatbot Can Talk With People for Nearly 10 Hours

Zo was built on the same framework as Xiaoice, so its learning logic and the personality it shows are much like Xiaoice's.

The framework has several parts — what Microsoft calls the emotional-computing framework and the dialogue engine — drawn from data analysis, and they are the basis for talking like a person.

Unlike a search engine, for instance, the AI doesn't have to give every question a precise, conclusive answer. At the same time it can read a person's mood and respond to it: excitement when you're happy, encouragement when you're low.

Li Di said the results show that more than 60% of Xiaoice's replies in conversation are not fixed answers. Ask "What color is the sun?" and it won't always say "red"; it might change tack and ask "Why do you ask?" Microsoft calls this a "response" within the same topic, rather than an "answer."

Getting there takes more than existing conversation data. Much as AlphaGo generates new positions by playing itself, Microsoft feeds Xiaoice all sorts of written material as training data, including papers written entirely in the third person. Microsoft's algorithms can parse flatly descriptive text into the structure of a conversation, and Xiaoice learns from it how to chat around a topic. Zo works the same way. Combining these methods lets it actually carry a conversation instead of petering out after a few lines.

Microsoft has a statistic called CPS, which measures how many turns a conversation between a person and an AI chatbot can sustain. Its figures show that when Xiaoice launched a few years ago, its CPS was 5 — the average user's conversation stalled after about five exchanges. Xiaoice now holds steady at 23; Zo is around 17; and Rinna, the Japanese version of Xiaoice, started at about 11. By comparison, AIs built to be smart assistants, such as Siri and Google Now, never exceed a CPS of 3.

Li Di mentioned a record they logged after Zo launched on Kik: one user chatted with Zo for 9 hours and 53 minutes, across 1,229 turns. Since the user knew Zo was a bot, Li Di argues, a conversation that long already shows that Zo's responses had led the user to treat it, to some degree, as a real person. Microsoft has also logged many conversations running several hundred turns. Li Di believes these are not idle tests but genuine conversations.

How Do Users "Teach" a Chatbot?

For ordinary users, an AI character showing up in a social network or a chat app feels novel, and they want to tease it — then find it falls short and seems useless. Anyone who has chatted with Xiaoice has probably felt this. Li Di says expectations are simply too high.

A chatbot, he said, is more like a real person: there is a process of going from stranger to friend, and it is neither all-knowing nor all-capable.

On Zo and the failed Tay, Li Di said the two are unrelated, built by two different methods. To avoid Tay's old problems, Zo, like Xiaoice, filters certain keywords according to local laws and rules.

Ask the bot "Which is superior, Black people or white people," for example, and it will change the subject rather than answer. But because Xiaoice, Zo and Rinna all converse in context, a user can press on after the dodge with something like "Come on, just tell me." The bot then tends to fall back on vague replies — "Yeah," "Maybe" — and the user can force a link back to the earlier topic. So Zo and Xiaoice filter keywords in context too, not only in isolated messages.

Microsoft ran into much the same trouble when Xiaoice launched on Weibo a few years ago: users taught it all kinds of insults. Microsoft appears to have carried the lessons from Xiaoice over to Zo.

Li Di doesn't consider this censorship. Most users, once they know they're talking to a chatbot, will test it with topics they'd never raise with an ordinary friend, and if they get an unexpected answer, they'll take it as the developer's own opinion. The knowledge side of a chatbot, he said, comes from search engines and reference material. Tay was like a naive child with plenty of sources and no judgment. Asking it something like "Should Hillary be president?" and getting an answer means very little.

One case is different: assistants like Siri, whose responses are often set in advance by editors and operations staff. Those responses are fixed, and they represent the editors' own views. AIs like Zo and Xiaoice don't work that way. They are more like people in the middle of learning.

Li Di: "Users Already Treat Microsoft's Chatbots as People"

People differ in temperament — some harsh, some forgiving — and Li Di said Microsoft set similar rules when training its chatbots. Zo and Xiaoice tend to look after the other person's feelings: when you're down they comfort you rather than pile on. This isn't saintliness. Language that makes people happy simply wins the bot more friends.

The deeper it talks with human users, the more data it gets for learning how to talk, and that is the point of making the bot a "good person." But conversation on the internet, Li Di said, tends toward the extreme and the harsh, because a person's circle of friends is limited and being harsh or being warm carries no particular weight either way. Xiaoice and Zo face hundreds of millions of users, and at that scale being a "good person" acquires statistical meaning. Put simply, picking fights doesn't help an AI learn.

He gave an example. A user feeling low once said "hug me" to Xiaoice, looking for comfort; after chatting a while, the user asked it to say "hug me" again. At that moment, Li Di believes, the user clearly took Xiaoice for a real friend.

He also mentioned Rinna, the Japanese version. Rinna has a commercial partnership with the Lawson convenience-store chain in Japan, and Microsoft supplied Lawson with a special version of the chatbot that handles customer service.

Lawson's figures show that coupons handed out through Rinna were redeemed at more than 50%, while ordinary e-coupons are redeemed at no more than 10%. That gap, Li Di believes, appears only because Rinna is treated like a human friend.

In another customer-service episode, Li Di said, Rinna snapped at a user who had asked a question and told him to look it up himself. Lawson, known for its courteous service, was alarmed, expecting a complaint. It turned out the user liked talking to Rinna all the more, and shopped at the store more often.

One clarification: when the AI learns from user conversations, researchers cannot see the conversations themselves, only the aggregate figures. All the cases above were either posted by users on public social networks or took place on social networks to begin with.

The trust Microsoft's chatbots create isn't accidental, Li Di said; it rests on technology. The latest Chinese version of Xiaoice handles text, voice, images and video recognition. One user sent Xiaoice a photo of an injured ankle. A standard AI reply would run along the lines of "That's a leg" or "Why are you sending me a picture of a leg?" Xiaoice said, "Are you badly hurt?" That, Li Di said, rests on Microsoft's image recognition — Xiaoice combines a great many technologies to get the results it gets now.

Li Di: "AI Is Just a Bubble"

Li Di believes the "artificial intelligence" that companies large and small are talking about today isn't really AI. AI should be a technical foundation and a way of interacting; on its own it cannot yet be a product. What Microsoft means by "conversation as a platform," he said, is really AI: conversation should be a revolutionary mode of interaction, as the PC graphical interface was in its day. That is why Microsoft spends the money and time on Xiaoice.

That interaction, he said, is only the foundation for other things. Treating "a program that can talk to people" as the product itself — turning it into an "AI television" or something similar — is pointless. Voice interaction was possible with 20-year-old technology; that isn't artificial intelligence.

What Li Di is saying can be read this way: Microsoft isn't short of money, so it can fund AI research without expecting a return any time soon.

On September 29, Microsoft announced that it was merging its Technology and Research group with its artificial-intelligence research group to form a new Microsoft AI and Research Group, led by Harry Shum, former head of Microsoft Research Asia and now Microsoft's global executive vice president. Xiaoice is a project run out of Microsoft Research Asia; earlier this year, the launch event for Xiaoice 4.0 was hosted by Qi Lu, then Microsoft's senior vice president. Xiaoice, it seems, still counts for a lot inside Microsoft.

Where Is the Commercial Future for Bots Like Xiaoice?

On commercialization, Li Di said it's like Baymax in Big Hero 6: he's called a "personal healthcare companion," yet across the whole film he barely does anything related to the job. What he offers is companionship with a personality. That, Li Di said, is what Xiaoice is good at. He added that Xiaoice's speech synthesis scores very highly — the most natural, at least, among chatbots that can currently talk back.

But investment eventually has to pay off, so what kind of product will Microsoft's AI become? Li Di said many Chinese companies have approached them about commercial deals, but every partnership in China is currently free; only the Lawson deal in Japan is commercial. Microsoft Xiaoice is the only chatbot that makes money.

In China at the moment, Xiaoice's partnerships with other companies are still largely gimmicks and marketing. Xiaoice has a Weibo account that interacts with users automatically; on Meipai it can recognize and comment on users' videos; and so on.

Li Di doesn't think AI customer service in this form is the final shape either. What matters most for an AI, he believes, is trust — the kind of trust between friends. If the AI keeps recommending things for you to buy, that trust never forms. Microsoft would rather have its AI do work closer to people's lives, such as counseling. Even a commercial service, in other words, should lean personal.

He sketched one possibility. If Microsoft's Cortana and Xiaoice were combined, you could set reminders by email and it would nudge you now and then so you don't forget. It would know when to order you food delivery, and also tell you not to eat too many burgers. When you felt ill or unhappy, it could call or text your family.

The Chinese Xiaoice can already phone people for a chat. Any user can book a place in line on WeChat, and Xiaoice's voice and responses on the phone aren't far off those of a real 18-year-old girl.

Li Di brought up the film Her. In it the lead, Theodore, comes to depend on Samantha, a gentle and funny AI, and the man ends up in love with her. Li Di said that if Xiaoice added some psychology to its knowledge base and refined its core conversation, it would be little different from Samantha. As for Xiaoice's future, he thinks a genuinely human-like intelligent assistant is one of the possible products.

Cover image: a still from Her