Netflix's New Ratings Math Sacrifices Clarity for Flashy Numbers

The Witcher-Publicity 3-H 2019

Netflix touted the runaway success of its series The Witcher in its quarterly earnings report Tuesday, noting that 76 million member accounts — 45 percent of the streamer's global subscriber base — "chose to watch" the Henry Cavill-led fantasy drama. The company says The Witcher is on track to have the biggest first season of any Netflix original.

Five paragraphs later in the quarterly report, Netflix revealed that "chose to watch" wasn't just a turn of phrase, but a new way it will reveal viewing data to investors and the public. The streamer says the new metric "levels the playing field" for titles with varying run times, but it also means less transparency in terms of how many people are seeing the full run of a film or movie.

Netflix now will count viewership based on just two minutes of viewing time — "long enough to indicate the choice was intentional," per a footnote to the quarterly report. The rationale behind the change is that it brings its viewership reporting closer to that of services like YouTube and the BBC's iPlayer, both of which it cites in the report. It also, however, makes the numbers nearly meaningless in trying to determine how many Netflix users actually watched a given title — as opposed to just checking out the equivalent length of time of a trailer.

"It's obviously to goose their numbers," one rival network exec tells The Hollywood Reporter. A second executive at a different outlet wonders why a publicly held company "who trades in 'viewership' isn't held to the same standard of independent verification as the rest of us."

Netflix wasn't exactly a paragon of data transparency before Tuesday — no streaming platform shares detailed viewer data. The old Netflix metric — a "view" meant a member account watching 70 percent of one episode of a series or 70 percent of a film — also didn't reveal a whole lot for series, as 70 percent of a single episode is likely less than 10 percent of an entire season. It was more useful for movies: If Netflix said (as it did) that 32 million member accounts worldwide viewed Always Be My Maybe, it's reasonable to assume that a healthy percentage of the people attached to those accounts finished the film.

But what if Netflix said 43.2 million members "chose to watch" the Ali Wong-Randall Park rom-com? (That figure comes from the company saying the new measurement results in numbers is "about 35 percent higher on average than the prior metric.") Two minutes of Always Be My Maybe is 2 percent of the film's one-hour, 40-minute run time.

It's an issue not just for Netflix, but across the streaming media landscape, industry analyst Matthew Ball tells THR: "These issues stem not from trying to define apples to apples, but ensuring a figure actually means something."

Ball notes that music service Spotify bases its counts on a 30-second stream, regardless of whether the song is "Old Town Road," which is less than two minutes long, or the Beatles' eight-minute recording of "Hey Jude" — and also regardless of whether a user finishes the song.

"Two shows can both get the same viewer numbers, but one actually drove a subscriber to add or keep a service, while another was just background filler," says Ball. "This is more important than, and mostly unrelated to, views. Having Taylor Swift on your music service matters more than hundreds of others, even if their aggregate plays are comparable."

The closest thing in traditional TV measurement to Netflix's new standard is Nielsen's "reach" statistic, which counts every person who watches at least six minutes of a given program. It's typically a good deal higher than the average audience for a show.

"They can do whatever they want," says the first executive about Netflix, "because they're not selling off of it." True, but most Netflix creative talent would like to know how many people are actually consuming their work. And linear TV's reach would likely look bigger too if the Nielsen qualifier were knocked down to two minutes.

But the reach numbers from Nielsen are often a way to gloss over a program's middling (or worse) average audience. In the case of The Witcher, the two-minute qualifier means those 76 million accounts could have watched no more than 0.04 percent of the season's 474-minute run time.

Is that actually what happened with the show? Almost certainly not. The point is that with Netflix's new math, it's nearly impossible to tell.