As the Rules Tighten, Which Internet TV Box Lets You Watch the Most? We Tested 1,560 Titles

Put simply, no box lets you watch everything.

How is your TV box doing?

Last month, many Tmall Box users found that the live-streaming apps on their boxes had vanished with the latest system update.

Under Document No. 229 from the State Administration of Press, Publication, Radio, Film and Television (SAPPRFT), 81 video-aggregation and live-streaming apps, including Fengyun Live and Xigua Yingyin, were ruled illegal and pulled. Only apps that work with licensed operators may now appear on the boxes.

You won't find these apps in the boxes' built-in app stores, and ones that came pre-installed may disappear too.

In June 2014, SAPPRFT told licensed internet TV operators to close the download channels for non-compliant video software on internet TV devices. In the two years since, the restrictions have only grown tighter: content from video sites must now reach TVs through an entity that holds an integrated content-distribution license. So far SAPPRFT has issued seven internet TV integrated service licenses, to CNTV (CCTV International), Hangzhou Wasu, Shanghai Media Group's BesTV and others.

There are still ways around the system restrictions. Connect a box to a computer by USB, for example, and those live-streaming apps can still be installed. But it is easy to see that getting paid content for free will only get harder.

This installment of QDaily Lab looks at what the internet boxes actually offer. We wanted to find out how further tightening would affect the video-site business, and how much a user who wants to watch TV on a box can really watch.

We chose seven boxes and one TV

Whether a TV box has enough to watch depends mainly on the video site behind it. For hardware makers such as Xiaomi, Huawei and WhaleTV, how much content their boxes carry also depends on their deals with content providers.

We tested three kinds of content: films, TV series and variety shows.

For foreign films, we used the global top 20 box-office titles for each of 2014 and 2015.

For Chinese films, we used the top 20 box-office titles in China for each of 2014 and 2015.

TV series and variety shows date faster, so we chose them by Baidu search ranking: the top 50 TV series and the top 50 variety shows.

Because the top 50 contains almost no American or Korean series, we added the top 10 Korean dramas and the top 10 American series from Baidu's search charts.

That makes 200 films, TV series and variety shows in all. Our method: search for each of the 200 titles on each box, and see whether it could be watched and whether it cost extra.

QDaily Lab bought several of the best-selling TV boxes on JD.com, plus a Tmall Box bought from Tmall.

WhaleTV, a new internet TV brand, sells only TVs and, unlike LeTV, has not released a standalone box, so we tested it on the TV itself.

The devices were:

• Xiaomi Box 3, 318 yuan;
• Huawei Honor Box, 298 yuan, working mainly with GICV and Mango TV;
• Himedia Mango Hi Q, 299 yuan, working mainly with Mango TV;
• Skyworth Q+, 199 yuan, working mainly with Tencent;
• Skyworth iQiyi Box, 299 yuan;
• LeTV New 1S, 499 yuan;
• Tmall Box 2, 299 yuan, with Youku Tudou and Tmall Cinema built in;
• WhaleTV 43-inch, 2,088 yuan, working mainly with Tencent and Sohu, plus some content it owns the rights to.

Here are the detailed results.

Foreign films are patchy on every box, even if you pay

On these boxes, the content that costs extra is mainly films. But don't expect to buy one box and watch the world's hit films.

We took the top 20 box-office titles of 2014 and of 2015. Even with a membership, you can watch at most about half of them.

For free viewing, Tencent ranks first. Most of WhaleTV's content comes from Tencent, so its free selection matches Tencent's.

Apart from Honor, which has no paid films at all, every box makes you buy a membership if you want more Hollywood blockbusters. If you pay, Tencent (Skyworth Q+) and iQiyi (also made by Skyworth) cover the most.

Cost aside, the Xiaomi box lets you watch the most, but only if you buy all three memberships Xiaomi offers (75 yuan a month in total) and then pay separately for new releases. Quite a few of those, such as Furious 7 and Minions, come with a single membership on other platforms. In short, it is poor value.

Popular Chinese films: you can watch half without paying

The China box-office chart overlaps somewhat with the global one. But on the boxes, more of these films are free.

WhaleTV does best here, with half the films free. Pay by the month and you can watch more than three-quarters of them.

Next come Huawei Honor, Tencent, Xiaomi and iQiyi. Honor has no paid tier, while iQiyi, Tencent and Xiaomi are much alike.

Variety shows: quite a few are only on Mango

Variety shows are the best-covered category on every box, and almost all are free. Even LeTV, which has the fewest, covers nearly 60% of the search top 50.

Mango TV does not have the most titles, but many of the variety shows that have drawn attention in recent years are made by Hunan Satellite TV, and Mango TV is the exclusive online broadcaster of many new ones.

If you want the latest season of I Am a Singer, the fourth, for example, you will find it only on Mango TV.

The one exception is the Xiaomi box, which carries Mango TV's programs but makes you pay 30 yuan a month for a membership to watch them.

TV series are also mostly free

Baidu's chart lists only the top 50, and two of its entries, New Xiao Shiyilang and Xiao Shiyilang, both point to the new version recently aired on TV stations and video sites, so we counted only the top 49 Chinese TV series by search popularity.

Chinese series dominate Baidu's top 50. Leaving paid content aside, WhaleTV offers the most, followed by Xiaomi and iQiyi.

Don't count on Korean dramas

Most of the top 10 Korean dramas by search popularity are not the newest. Descendants of the Sun, the hit iQiyi imported exclusively, did not even make the chart.

Even so, coverage is low on every box. iQiyi and WhaleTV, the best of the lot, cover only 30% of the top 10.

For recent American series, your choices are basically Tencent, LeTV and WhaleTV

The rights to American series sit mostly with the LeTV and Tencent boxes. Tencent bought the rights to eight of the top 10 American series by search popularity, and LeTV bought five. WhaleTV, which plays content licensed by its partners Tencent and Sohu Video, covers only 60%. The Xiaomi, iQiyi and Huawei Honor boxes and Mango TV's box cannot play any of these popular American series.

If a series had only older episodes and not the latest season, we counted it as "not available."

American series have faced more and more restrictions in the past two years, and many of the latest hits, such as Daredevil and Jessica Jones, have not been imported at all.

The days of buying one box and watching everything are over

TV boxes are no longer what they were when they first came on the market. Back then, you bought a box and could watch almost every film and TV program free, and receive cable channels through it as well.

Today's boxes charge much as video sites do. Some content is free, but for the newest and hottest films and shows you still have to buy a membership.

So our experiment found that every video platform has some resources of its own, but no box meets the needs of every category.

If you want plenty of imported films and American series, the Tencent-Skyworth Q+ box and WhaleTV suit you best. Without a membership, of course, you get only classic films and older American series. A Tencent membership is still worth buying.

LeTV and iQiyi have fewer titles, but each has invested heavily in exclusive series and its own productions. Some exclusives you can watch only on a particular box: The Legend of Zu and Descendants of the Sun are on the iQiyi box, while Go Princess Go, The Legend of Mi Yue and Empresses in the Palace mean switching on LeTV.

If you only want Chinese variety shows and Chinese TV series, the Honor box is the best choice; beyond those, it has nothing. Mango TV's box may suit fans of Hunan Satellite TV better.

Xiaomi is a special case. Its three memberships cover the most content, but at 75 yuan a month combined, they cost roughly two to three times what competitors charge.

Beyond content, a few other things worth mentioning

• Xiaomi Box 3

Xiaomi has the most complicated payment system of all the boxes we tested: a Mango TV membership at 30 yuan a month, a Lizhi membership at 30 yuan a month, and a CNTV Youdian Hollywood membership at 15 yuan a month. Even the most expensive Xiaomi TV comes with no membership. By our results, Xiaomi, the first Chinese company to make boxes, is also the one hit hardest by the policy.

But the Xiaomi box has its merits: its software runs smoothly, and its design and build are close to the best of the lot.

On the downside, although you can use a phone as the remote, the phone only imitates the remote's arrow keys; you cannot type a search with the phone's keyboard.

To search for and play videos from your phone, you have to download a separate app, its "screen-casting tool."

• Himedia Mango Hi Q

Mango TV launched this box with Shenzhen-based Himedia. A Mango TV membership costs 15 yuan a month, and some films carry an extra pay-per-view fee.

If you are a fan of Hunan Satellite TV's own variety shows, this box is all you need. But you can barely watch any American or Korean series on it. One handy feature: you can bind a WeChat official account and use it as a remote.

• LeTV New 1S

The box comes with six months of LeTV Super Movies and TV membership, which also covers PC, tablet and phone; renewing month by month costs 42 yuan.

Of all the boxes, LeTV's feels most like a TV when you switch it on: you first see programs playing in rotation rather than the menu screen common on boxes.

As on the Xiaomi box, the maker's own products are promoted all over the place. The two companies, both trying to build an "ecosystem," are much alike in this respect.

We found no voice-search button on the remote, though the system says voice search can be turned on. You can, however, use the multi-screen feature to push a video straight from the LeTV app on your phone to the TV.

• Skyworth iQiyi Box

The box is not cheap, and neither is the membership, at 29.9 yuan a month.

Once you activate a Galaxy Movies membership, the same account lets you watch VIP content in iQiyi's phone, computer and tablet apps, but it still costs considerably more than an iQiyi VIP membership.

• WhaleTV 43-inch

The TV comes with one year of membership; renewal after that costs 199 yuan a year. From what we saw, WhaleTV's content comes mainly from Tencent, with some from Sohu Video. Its voice search works well, probably because it has a built-in vocabulary of film and TV content.

Of the eight products we tested, WhaleTV is the most demanding on the network; on a slow connection, nothing may open at all.

• Skyworth Q+ Box

The box comes with a 90-day Penguin Cinema VIP card; renewal costs 25 yuan a month. The software is sluggish and tends to stutter.

Although it draws on the same Tencent source, the Q+ box's library is smaller even than the Tencent content WhaleTV carries. Some blockbusters require a Penguin Cinema membership, but the two memberships are not interchangeable: some films that Hollywood members can watch in the app will not play on the Q+ box.

• Tmall Box 2

The box comes with five days of Tmall Cinema membership; a month costs 39 yuan, and a year 365 yuan.

Tmall's problem is that, even in its second generation and after several rounds of system changes, it is plainly still an unfinished product. Some Youku videos, for example, cannot be found through search; you can reach them only by browsing the lists.

And although it can connect to a Bluetooth keyboard, the keyboard cannot type into search; it works only like a remote, through the arrow keys.

• Huawei Honor Box

This is the only box in the test that doesn't expect you to buy a membership. It has no paid content at all. If you watch only Chinese TV series and variety shows, there is little to complain about with the Huawei box; but if you often want foreign films and TV, don't buy it.

Getting into the living room is hard for video sites

Internet TVs and TV boxes are no longer a hardware business. Content decides the TV experience: what matters most is whether you can find the program you want when you switch on the box.

Xiaomi, the first Chinese company to make boxes, with the largest installed base and the most polished product details, has fallen behind LeTV, WhaleTV and others on content.

And given the new policies and commercial competition, finding everything in one box is now all but impossible.

From our results, the companies with the widest coverage either bought a great deal of licensed content, like Tencent, WhaleTV and iQiyi, or spent heavily on their own productions: Mango TV, backed by Hunan Satellite TV; iQiyi, which made U Can U BiBi a hit; and LeTV, whose Go Princess Go drew many paying members.

But licenses burn money, and over the past two years video sites have spent lavishly on program rights.

iQiyi's recent import of the Korean drama Descendants of the Sun, for example, cost as much as US$230,000 an episode. As early as 2014, the company spent more than 200 million yuan on variety-show rights.

Tencent, the richest, also has the biggest library. In TV series, Tencent Video has secured broadcast rights to dozens of shows, including Zhu Xian, Ice Fantasy and The Legend of Mi Yue. By 2016, Tencent's film library had passed 7,000 titles. Our results bear this out: in both films and American series, Tencent itself offers the most.

The video sites' hunger for content only drives licensing costs higher. Every platform wants the rights to a hit series, so rights holders naturally raise their prices.

At this point, the model Chinese video sites relied on in their early years, free streaming paid for by advertising, can no longer keep up.

According to the latest quarterly report from Baidu, iQiyi's parent company, iQiyi lost 3.65 billion yuan in 2015.

Bear in mind that Baidu is still China's largest Chinese-language internet advertising platform, and the user data it has collected for years has long been linked to iQiyi, an advantage no other video site can match.

Today, video sites have all turned to Netflix's model: offer exclusive original content so that users stay for it and become paying members. The recently released House of Cards is a good example.

China's video sites understand this just as well. The membership growth that The Lost Tomb brought iQiyi and Go Princess Go brought LeTV are ready-made examples.

But running a paid business in China is not as easy as in the US.

For one thing, American users got used to linking a credit card to a paid service, billed automatically every month, back in the days of magazine and newspaper subscriptions.

The Chinese market has never taken to automatic payment. The video sites' challenge has shifted from persuading users not to leave to persuading them, every month, to pay once more, and that is much harder.

For another, Netflix costs only US$9.99 a month, less than a tenth of a typical broadband-and-cable bundle. Users can add it on without a second thought.

In China, cable TV usually costs only a dozen or so yuan, far less than any video site's membership. With that incentive, it is no surprise that people share video-site accounts everywhere.

Membership fees also clash with advertising: the people most willing to pay for a better experience are taken out of the ad audience.

LeTV and WhaleTV are trying to solve the problem by bundling memberships with hardware. Buy a TV and you get a year's membership free, in the hope that once you are used to watching paid content, you will keep paying for it.

But whether a company runs only a video site or sells TVs too, to persuade users to renew month after month it must offer enough good content.

And good content is hard to come by, whether you make it yourself or pay heavily to buy it. Not many companies will last to the end.

Cover photo: Ma Ruofei; illustrations: Cui Qiwen

Thanks to Huang Junjie, Ma Ruofei, Tang Yiyuan, Zhao Changcun, Zhou Shaohong and Zhong Shuting for their contributions to this experiment.