Living With Xiaomi's and Alibaba's Smart Speakers: the Next "Entry Point," or Just Toys Like Siri?

We lived with Alibaba's Tmall Genie and Xiaomi's Xiao AI for a few days.

"Xiao AI, play me a Lei Jun song."

"Sure, playing Lei Jun's greatest hits."

A few seconds' pause, then an unusually loud voice: "33333Q! Are You OK?!"

The Xiaomi AI speaker on the desk, which looks like a miniature air purifier, had started playing "Are You OK," a meme remix cut together from footage of Lei Jun's speech in India.

The first time you meet a smart speaker, it is hard to resist asking it strange questions, just to see how smart or how dumb it is.

It turns out to have a Lei Jun playlist, so the next question is obvious: "Xiao AI, who's your boss?"

This time it doesn't understand.

It is like 2011, when Siri first appeared on the iPhone. Even though it kept telling you "I'm not sure I understand," its preset, out-of-nowhere wisecracks could still delight you: so it knows this one too.

Today not many people have built a habit of using Siri. Once the novelty wears off, the frustration of "I don't understand" kills your enthusiasm. In the end, it just isn't much use.

The voice assistants tied to phones a few years ago all failed, so technology companies turned to speakers that work on their own. This trend, started by Amazon's Echo two years ago, has now reached China. Xiaomi's AI speaker went into public beta in August. A month earlier, Alibaba's smart speaker, the Tmall Genie X1, did the same. Startups including Rokid and Mobvoi have already shipped their own.

The story told about these speakers is much the same everywhere. It plays music straight from internet services, controls Xiaomi's Mijia smart appliances with a single sentence, or places an order on Taobao. It is an entry point.

But a story is only a story. If something isn't useful, people forget it, the way they forgot the Siri built into the iPhone.

So are these speakers useful? Over the past few weeks we spent more than a hundred hours talking to Xiaomi's and Alibaba's speakers.

"Xiao AI, do you think you're pretty?" "Um, I guess. I don't think I'm ugly."

Because of how it picks up sound and plays it back, a speaker is something you put in the middle of the living room. So how it looks, and how it is designed, still matters.

The Xiaomi AI speaker, still in public beta, looks unmistakably like a Xiaomi product: white, plain in shape, plastic shell. For a speaker it doesn't feel heavy in the hand; the bottom is ringed with irregularly placed sound holes, and the boxy body — apart from the slightly rounded top corners — makes it look like a mini Xiaomi air purifier.

Plug it in for the first time and the LED ring around the top edge starts pulsing a soft blue. In use, the color of that ring is part of how the speaker talks back to you. After a few seconds it turns orange: pairing mode.

The speaker hasn't officially gone on sale, so Xiaomi has stuck a QR code on it. Scan it and you can download the Xiaomi AI app, still in testing, and pair your phone with the speaker.

Once the app is installed, pairing goes smoothly. The phone finds the speaker over Bluetooth on its own, then asks for your home Wi-Fi password; enter it and the speaker connects by itself. That is much easier than the Amazon Echo, which makes you first connect your phone to a Wi-Fi hotspot the speaker puts out.

Part of that ease comes from the fact that it only supports Android phones for now, where apps can control Bluetooth directly. On an iPhone, which limits apps more, the process would be different.

The control area on top looks far too much like an iPod click wheel. It carries four touch buttons — volume, skip track, microphone mute and pairing mode — but pressing them produces no vibration and no sound, so you are easily left wondering whether you pressed correctly.

A touch strip rings the buttons; you slide along it to change the volume. There are too few bulbs in the LED ring, so your finger has to travel some distance before the volume visibly moves.

In short, anything on the Xiaomi AI speaker that requires touch is done poorly. But once you get used to talking to it, you barely touch the buttons at all.

"Xiao AI" is both the Xiaomi AI speaker's nickname and its wake word. Once woken, its light ring comes on and points toward the direction your voice came from. If the network lags, the ring at least tells you the speaker heard you.

Every time you want to talk to it you have to start with the wake word, "Xiao AI," and every time it answers: "Yes, I'm here!"

And you have to wait for it to finish that line before you can give the next command.

"Xiao AI, play a different song." "Sure, now playing 'Different' (《不同》) for you."

Playing music is, of course, the most important thing a speaker does.

Bluetooth speakers could always play songs from services like QQ Music, Xiami or NetEase Cloud Music. But you had to pick the song on your phone.

With today's smart speakers, if you get home from work, drop onto the couch and want some noise in the house, or you need background music to set the mood at a party, a few words to the speaker takes care of it.

Talking to a smart speaker is much simpler.

"Play me a song" — it understands that. Though the song it picks at random may not be to your taste.

If you want something more precise, it takes a little skill.

The most direct way is to say clearly the name of the singer or the song you want; as long as it is in the library, it plays. Try asking Xiao AI for Lei Jun's other track, "MI, MI, MI."

Xiao AI also recognizes genres — jazz, rock, pop, country — and you can ask for a mood: say "play something upbeat" and a few seconds later you hear, "Follow me, left hand, right hand, one slow-motion move; right hand, left hand, slow-motion replay."

Once your request gets vague, most of what Xiao AI picks comes straight off the top charts of the music library it is connected to.

Xiao AI draws on the libraries of several music apps, including Xiami Music and Xiaomi Music. But you can't import the playlists you saved in those apps into the Xiaomi AI app. If you are a longtime user of one of them, used to favoriting songs there, don't expect Xiao AI to have learned your taste from it, at least not yet.

On music controls, Xiao AI recognizes volume up and down, pause and play well. But as with picking songs, you have to say it outright: "Xiao AI, pause playback."

Smart speakers pick up some everyday spoken phrases. If the song is truly terrible, try shouting "shut up." We also told it once, "Xiao AI, this is awful," and it understood.

But plenty of commands that are not at all complicated for a human trip up a smart speaker's grasp of meaning, which produces some comedy.

Tell Xiao AI to "switch to a different song" and it starts playing a song called "Different." Say "Xiao AI, switch songs" next, and it switches to another song also called "Different."

The easiest way to actually change songs is to say, "Xiao AI, I don't like this song."

On sound, the Xiaomi AI speaker claims a large bass chamber and a bottom reflector cone, but in practice it sounds about like what 299 yuan buys.

Overall, if you just want music to relax to when you get home, or a bit of music at a party or a game night, the smart speaker's music playback clears that bar.

And you don't have to worry about a message alert or a shrill phone ring suddenly cutting the music off.

"Xiao AI, besides singing, what else can you do?" "Sorry, I can't find a song called 'What Else Can You Do?'"

The voice assistant inside a smart speaker brings to mind JARVIS, the virtual assistant in Iron Man: you talk to it, and it handles everything under the sun for you.

For now, you can't ask that much of a smart speaker.

Every smart speaker on the market keeps adding services — traffic, weather, radio — to seem smarter, but the bottleneck is still understanding what you mean, and the details of each feature fall short somewhere.

• Alarms and reminders

A genuinely useful feature: you can have it wake you at 8 a.m. tomorrow, or remind you to take off your face mask in 20 minutes.

But it can only set a reminder from the literal words. Ask it to remind you about tomorrow's 8 a.m. flight and it will simply tell you at 8 a.m. to catch your flight; it won't, as Google's speaker does, pull up your flight number and use traffic conditions to tell you when to leave.

You also can't delete an alarm by voice. Try to cancel one you set by mistake, and all you hear is that the alarm has been set, again.

A phrasing it understands: "Xiao AI, remind me to take my face mask off in 20 minutes."

A phrasing it doesn't: "Xiao AI, don't remind me to take my medicine at 10 p.m."

• Unit conversion

You may not need conversions often, but asking Xiao AI at least saves you opening a phone or a computer to search. The same goes for exchange rates: checking the local rate while shopping online is handy.

It still can't recognize Bitcoin, though.

• Looking things up

You can ask what something is, and it can't always tell you. The answers Xiao AI reads out come from the title and first paragraph of the matching Baidu Baike entry. Given the quality of Baidu Baike, that is not a good experience.

• Checking the weather

Xiao AI looks up the weather quickly: not just cloudy, sunny, rain or snow, but also the air quality and PM2.5 readings worth paying attention to.

Only, when the PM2.5 index is around 70, it tells you "there's no smog in Beijing today" rather than the number.

The weather feature is also fairly smart — it understands "will it rain this afternoon?"

The weather may also be the answer you hear most often. Ask something it can't yet handle, like "what popular movies are showing in Beijing right now," and its response is to read the weather out again.

• Checking traffic

Xiao AI is the only smart speaker that checks traffic. The feature comes with plenty of limits.

In our tests you first have to set home and office addresses on your phone. Only then will it give you transit or driving times, and rough arrival times, from those two fixed starting points. Ask about the traffic between any other two places and all Xiao AI can do is read out the travel time from home to them.

It also can't give you a traffic report at a set time each day. All you can do is ask it before you go out.

A phrasing it understands: "Xiao AI, how long does it take to get from home to Wangjing?"

A phrasing it doesn't: "Xiao AI, how long does it take to get from Sanlitun to 798?"

• Stock prices — better to look them up yourself

Stock lookup is even less finished. Xiao AI can only read out a price and a change.

Ask for Kweichow Moutai's share price and it says: "Kweichow Moutai closed at 492 yuan, with a change of 0.94%." There is no way to tell whether the stock went up or down.

We also asked about LeEco, whose shares had been suspended from trading for a long stretch. Xiao AI correctly found Leshi Internet, and during trading hours it will tell you the stock is suspended. Outside trading hours it just reads out the price from long before the suspension, and says nothing about the halt.

Indexes investors watch, like the S&P 500, it doesn't understand either.

A phrasing it understands: "Xiao AI, what's LeEco's stock price today?"

Other features:

• Jokes, neither funny nor lame enough to be funny. "Xiao AI, tell me a joke that isn't too lame?"

• News: it plays news audio straight from Ximalaya. "Xiao AI, what's the latest news?"

• Keeping a child happy: it imitates elephants, pigs and other animals. Not all of them. "Xiao AI, what sound does an elephant make?"

From what we tried, the features we used most were alarms, reminders and conversions. As long as the command is clear, they all work.

Weather, dictionary entries, news, radio and jokes basically work too, but the content is limited by the platform or the scraped website behind them.

Useful or pointless, every voice command has to be phrased exactly the way Xiao AI can understand before you get a clear answer.

Get it wrong a few times and the frustration builds.

These speakers have to learn in order to master a skill. And you, the user, have to learn too, just to make yourself understood. So it is still a geek's toy.

"Xiao AI, turn off the air conditioner in two hours." "Sure, the air conditioner is off."

China's most complete smart home system today is Xiaomi's Mijia. Companies Xiaomi has invested in make dozens of smart home products, from power outlets and air conditioners to purifiers and rice cookers; add the partner manufacturers that plug into the system and Mijia offers hundreds of devices to choose from.

Since smart home gear is not much use right now, Xiaomi's products, most of them priced at a few tens or a bit over a hundred yuan, are better suited to trying the idea out. One Nest camera that works with the Amazon Echo costs about as much as 20 Xiaomi smart home products.

The devices we tested were a dimmable Philips desk lamp, a Xiaomi air purifier, a Mijia rice cooker, the Mijia AC Companion (which can control any air conditioner) and a smart outlet.

All of them have to be connected through Xiaomi's own Mijia app first, then authorized for the Xiaomi AI app, before the speaker can control them.

For most products, what the speaker controls is on, off and simple mode switching.

The air conditioner, for instance: one sentence turns it on or off, or switches between cooling and heating, without reaching for a phone or a remote. But the scheduled on and off timers the phone app already offers are still beyond the speaker.

The lamp is the same. One sentence turns it off, but you can't switch on a specific function such as eye-care mode.

The purifier will go into sleep mode and slow its fan on command. But it already switches itself on and off based on the PM2.5 level, which is far more convenient than telling the speaker.

Voice control of the rice cooker is just as redundant.

In real life, cooking rice means putting in the rice and water yourself, and while you are there you may as well flip the switch and start it. There is essentially no situation that calls for voice. Remote start and scheduled cooking are far more practical than switching by voice between the cooker's precision and quick modes. And since the cooker can't tell whether anything is inside it, if you tell it to start cooking it will heat an empty pot.

In short, everything the speaker can do by controlling a device is a subset of what the smart home app on your phone already does. The only difference is that you don't have to open your phone.

Xiaomi's official promotional video imagines it this way: you wake up in the morning, say "good morning" to the speaker, and Xiao AI reads out the weather, opens the electric curtains, starts the water heater, tells you the traffic and the day's driving restrictions, and even switches on an egg cooker to help make breakfast.

All of that used to be done by hand. With a scattering of smart home products, you end up picking up your phone more often and tapping a pile of buttons in the matching app, which isn't especially convenient either.

As a command center for particular moments, Xiao AI could automate the on/off state and the working mode of these devices and save you a lot of that effort. But the feature hasn't been opened up yet.

And when a lot of Mijia devices are connected at once, Xiao AI's automated scenes stand a fair chance of failing to control some of them.

Turning off a light or the air conditioner with one sentence is convenient. It does not make today's not-very-practical smart home much more practical.

"I want the one with free shipping." "You haven't answered my question, so I'll exit for now."

Tmall Genie is a smart speaker built by Alibaba Cloud's AI Lab. Either "Hello Tmall" or "Tmall Genie" wakes it, and you can use it to buy things on Taobao by voice.

Saying either phrase out loud feels a little strange, and if more than one person is home you may find it awkward.

There is nothing remarkable about this all-black cylinder beyond being small and light. For a speaker that sits at home rather than travels with you, being very light usually just means poor sound.

Unlike the Xiaomi AI speaker, Tmall Genie's LED ring is hidden low on the body. It flashes white when the speaker wakes, but there is no sound.

Because there is no audio feedback at all, during our testing we often had to look over to see whether it had woken. That counts as a lapse in the product design.

Tmall Genie's body is likewise ordinary plastic. The top uses a six-microphone wake array with a mute button at its center, and the cavity holds only a single 5W driver, so the sound is not good.

If you use a 5GHz Wi-Fi network, setting up Tmall Genie will probably give you trouble. The speaker only supports the 2.4GHz band. The engineers explained that 2.4GHz gets through walls indoors better than 5GHz, so they used a single-band Wi-Fi chip. That doesn't make much sense.

Without a 2.4GHz network, you have to turn on a hotspot from your phone to use Tmall Genie at all.

Its headline feature is shopping. To solve the payment problem, Alibaba Cloud's AI Lab built voiceprint payment for Tmall Genie, so you can order and pay by voice.

But what you can actually buy is the real problem.

In practice, simply choosing a product can be deeply frustrating. Say, "Tmall Genie, I want to buy some snacks." The assistant answers: "I recommend the three-can pack of Lay's chips from Tmall Supermarket, 19.9 yuan, shipping not included for now."

But Tmall Genie didn't tell me what flavors the three cans were. Testing other categories produced much the same result.

Laundry detergent, for instance. The first time, Tmall Genie picked an Omo detergent for me, without free shipping. When I said "switch to one with free shipping," it replied at once: "You haven't answered my question, so I'll exit for now."

Tmall Genie can recommend a product from what you say, but it neither describes the product (color, style and so on) nor lets you narrow the choice by voice.

Take mountain bikes. I told Tmall Genie I wanted to buy one, and it first recommended a 27-speed Forever in red and black. Say "switch to another color" and it swaps the bike entirely, recommending a Giant instead.

In other words, voice shopping on Tmall Genie can neither filter for free shipping nor change a product's specifications. Whatever the cloud recommends is what you get. And with no idea of the specifications, almost nobody will place an order.

At Tmall Genie's current level of intelligence, for the great majority of products the most it can do is put something in your cart. Finishing the purchase still means opening your phone.

Standardized goods, socks from Uniqlo say, might suit direct purchase. But when a male reporter said he wanted a pair of Uniqlo socks, it recommended a set of women's socks.

Tmall Genie is already tied into Taobao's account system. Log into the Tmall Genie app with your Taobao account and all your purchase history, searches, favorites and other data should in theory be available to it. In practice it isn't.

To make sure nobody can place an order on your behalf, Tmall Genie asks you to record your own voiceprint before it will pay: "Tmall Genie" plus four random digits.

Give it the command "Tmall Genie, top up my phone with 100 yuan" and it confirms whether you want 100 yuan added to a number beginning 1xx. Once the number checks out, it moves to voiceprint verification.

You need a quiet room, and then you say "Tmall Genie" plus the four digits it prompts you with. When verification succeeds, it deducts 100 yuan from your Alipay account, following your default payment order.

Across several attempts to top up my phone this way, voiceprint verification was where things went wrong most often. It is very sensitive to background noise: a cat meowing indoors or a robot vacuum running was enough to spoil it.

Besides phone credit, Tmall Genie can order food delivery. Though you would probably only get used to this kind of delivery if your taste buds had stopped working.

At QDaily's Beijing office, for instance, it recommends Tingshangke Beef Soup near Guomao, 31 yuan after the discount, billed as the best-rated delivery nearby. If you don't want beef soup, Tmall Genie won't let you switch to another dish from the same restaurant.

The one genuinely convenient thing is ordering yesterday's meal again. If you eat the same delivery every day, that's handy.

Beyond playing music, they are still mostly geek toys

The smart speaker boom comes mainly from the Amazon Echo's unexpected success.

Morgan Stanley analysts estimate that between the middle of 2015 and December 1, 2016, Amazon sold 11 million Echo speakers, more than 10 million of them in the United States.

That is roughly one in every seven or eight American households.

No new kind of smart product has succeeded like that since the iPad.

After that success, Amazon turned the Echo into its story about the future of artificial intelligence, casting it as an entry point that doesn't depend on the smartphone. Its ads began to talk about shopping and about the smart home. Thanks to the Echo, Amazon even pushed forward its music service, which had gone more than a decade without much progress.

Smartphones have essentially stopped growing, so a new entry point means new growth. From Google and Apple to big Chinese companies — JD.com, Alibaba, Xiaomi — down to startups of every kind, everyone is in the smart speaker business.

From the basic shape to the companion phone app to the light ring and even the colors it glows, you can see Amazon's shadow on all of these Chinese speakers.

Building a smart speaker isn't hard: China has every element of production one needs. Microphone arrays, full-range drivers, far-field recognition algorithms, natural language processing — the designs and the suppliers are all there.

These speakers also beat Bluetooth speakers for playing music. One sentence starts a song, and a WeChat notification won't interrupt it. The rough edges in today's experience will be smoothed out by software updates too.

Building a smart speaker that does more than play music is another matter.

In the smart home, what a speaker can do is still very simple. And a smart home worth having shouldn't need your attention at all: the purifier turns itself on when the smog thickens, the curtains open when you get up.

The smart home platforms these companies run still require you to spend your own time setting up one product after another, like stacking blocks — hours of fiddling to save a minute a day later. Not many people will do that.

The speaker only removes the step of tapping a screen. Turning off a light that way is reasonably convenient. But the smart home's real problem has not been solved by the speaker, and it will not be.

As for shopping, scrolling Taobao is a habit for many people, and if you want speed, the Taobao or JD.com app is still fastest.

You will only use the speaker to "shop" when you feel like playing with it, or letting it play with you. And as with Siri, the fun of playing with a speaker passes quickly.

In fact, an earlier survey of Amazon Echo users found that out of its thousand-plus skills, only three were used by more than a quarter of users: playing music, turning lights on and off, and setting alarms.

Everything else is still a geek's toy. Like Siri, these features excite people for a while — but only for a while.

Cover image: photographed by the author