stdout.be

A blog about programming, information architecture and journalism

in journalism, business & management

Your metrics suck

Summary News organizations are not tracking the right things.

There’s a story floating around about early Twitter, back when they only had a couple of hundred users. Possibly apocryphal. The way Eric Ries tells it, their tiny user base was so embarrassing to them that they decided to offer a bunch of investors their money back, or what was left of it. “This isn’t working,” they must have thought.

More than one investor decided to cut their losses and took their investment off the table.

Thing is: if anybody would have bothered to look at Twitter’s per-user numbers it would have been immediately obvious that Twitter was destined to be a success. Sure, they only had a couple of hundred users, but those users were using the service obsessively. Obsession is a good indicator for a product that works. Twitter just needed to find a way to grow the product. And they did find a way to grow it, and those early investors who backed out are probably still kicking themselves in the head.

(If intrigue is your thing, another account says Twitter knew full well that they had something special in their hands, and used poor initial adoption as a way to trick investors to back out, leaving more stock in the founders’ hands.)

Early Facebook was different. Zuck didn’t care about pageviews, he cared about how many people with a Facebook account were active on a daily basis. He cared about likes, not pageviews, if you will. If you have active, loyal users, your customer lifetime value is going to be huge, meaning you can mess up in hundreds of different ways and still turn a profit. You can spend tons of cash on acquiring new users and still come out ahead. You can worry about monetization later, which is what Facebook did: if your users stick around, there’s no rush. Optimizing for retention is what allowed Facebook to become what it is today.

The news industry is like early Twitter when we should be like early Facebook.

Silicon Valley startups have gotten incredibly savvy about making data-driven decisions. The news industry has not. This post is the first in a series that hopes to translate the Valley’s wisdom into something that makes sense for journalism. Posts number two and number three are already up.


2 comments

Well put, Stijn.

I've observed that journalists and media people in general tend to optimize our output for the way in which we use it: As news junkies and information obsessives who are constantly plugged in, scouring the networks for something new. We optimize for so-called "power users."

We tend to forget that most people are not like us. We tend to forget that the fact that it's our job to find and process information quickly, in large volumes, and to keep feeding the beast — and to obsessively do so first, in a physical media rote behaviour throwback — makes us weird, anomalous outliers, and that most people just aren't like that. They just want to check in now and then, get up to speed on what they need to know, and be alerted once in a while when there's something that's really important to them.

Instead, we aim for and encourage fast-twitch, reflexive behaviour that's akin to a first-person-shooter video game, when it should be more like a massively multiplayer real-time strategy or role-playing adventure.

We abuse the real fast-twitch opportunities (e.g., breaking news alerts) by cynically wasting them on frivolous link bait: Do I really, urgently need to know that U.S. Olympic gold medalist Ryan Lochte said he urinates in the pool? Or that the couple photographed kissing during the 2011 Vancouver riot gave an interview? That's not news judgement, that's tactical instead of strategic business metrics — and flawed, at that.

We need to treat our news community — the people formerly known as the audience — as partners rather than consumers, a part of our process, and make that engagement ongoing rather than one-off experiments or following the old model of news as a finished product.

I can't recall whether we touched on this in our business models discussion in the Berlin sunshine last September, but if you can make it to Toronto in about three weeks, I think we can pick it up in a similar fashion.

To Saleem: agree wholeheartedly (and hope I can make it to Toronto.)

Just a small addendum. I believe retention is a better metric than pageviews for both Facebook and news organizations for the reasons outlined above: continually driving new visitors to your site is usually harder than making a product that keeps existing visitors around.

But there's a bigger lesson here. We should think really hard about what makes our business tick, set goals for where we need to be and choose metrics that help us reach those goals, not just randomly pick a couple of numbers and assume they're a good measure of success.

That's ultimately what set early Twitter and early Facebook apart – though I know I'm idealizing their stories to some extent. Facebook picked its metrics after careful deliberation of what sort of service it needed to be, Twitter because their metrics sounded reasonable. Picking metrics just because they look reasonable or commonsensical isn't good enough.