Proxies Drift

What is the number standing in for?

The idea

A metric stands in for something you care about, and the two separate once people optimise the metric.

In the real world

Time on page as a proxy for value, improved by making content harder to read.

Going deeper

A proxy correlates with what you care about while nobody is pushing on it. Optimising the proxy directly is exactly the push that breaks the correlation.

Time on page as a quality proxy rises when content is made denser, navigation harder, or articles split across pages โ€” all of which worsen the underlying experience. The metric improves and the thing it represented degrades. Naming the outcome a metric stands for, and how they could diverge, is the check that catches this before a quarter is spent moving the wrong number.

Where it stops applying

All measurement uses proxies to some degree, and refusing to use them leaves you with nothing. The discipline is periodic validation against the real outcome, not abandonment.

Why it matters

It is why a dashboard can improve steadily while the thing it represents does not.

Try this today

For one metric you target, name the outcome it stands for and how they could diverge.

Test yourself

A team targets time on page as a proxy for content quality. It rises 30% and satisfaction falls. What happened?

Show the answer

The proxy was optimised directly, and the easiest ways to increase it โ€” denser layout, harder navigation, split pages โ€” worsen the underlying thing. The correlation held while nobody was pushing on it and broke as soon as someone did.

Learn this in the feed Answering from memory, then again days later, is what makes it stick.

More in Data & Statistics

What a p-value Says Axes Can Lie Test Enough and Something Wins Regression to the Mean The Average Hides the Shape Simpson's Paradox

All Data & Statistics lessons