Sophie Alpert

Metrics by proxy

It’s usually hard to measure the things we care about. So we compromise.

What makes a metric good? In an ideal world, you could pick the thing you want to make better (say, “How productive are my users?”) and then measure it precisely. But since it’s rare to be able to take realistic measurements in reply to such a question, we need to come up with something else that we can actually measure. If you look around, you’ll find that most metrics we use are only an approximation of the real thing we care about. So I’ve been calling these proxy metrics.

I’ve noticed this pattern in several places, so in this post I’ll walk through how this works from multiple angles.

The technical side: Let’s take web performance. We care about how long things take on real users’ devices. But it’s hard to track that. Even if you set up the telemetry to collect performance numbers, they’re usually noisy.

The management side: Let’s say you’re running a company and you’re trying to figure out how to evaluate your employees. Your ideal metric might be, “How much value did this employee generate for the company during this time?”

The engagement side: About those product metrics I mentioned. Let’s say it’s 2004, and you’re building Facebook. How do you know if you’re doing a good job? Maybe your first instinct is: if people use the site more, they’re probably getting value out of it. (Note: This might not be how things actually happened. I work at Facebook, but not on this.)

At the end of the day, you’ve created some formula that purports to measure how meaningful people are finding their interactions. The formula looks silly. Dry.

Could it possibly know what people are feeling? Well, it actually kinda does! Because your user research told you that people value comments twice as much as likes. And so your formula is your best attempt at actually codifying that into science. It’s not perfect, but it’s based in human interaction.

It’ll have to do. At least until your next big realization, when you’ll pick a better metric.

(Thanks to Ada Powers for reviewing.)