Data & Statistics
Samples, spread, significance, and how honest numbers mislead
-
The Average Hides the Shape
A mean describes the centre and says nothing about how the values are spread or whether the centre is typical.
-
Variance Matters as Much
Two processes with the same average can behave completely differently if one is far more variable.
-
The Sample Decides the Answer
A result describes the group you measured, and generalises only as far as that group represents the wider one.
-
Regression to the Mean
Extreme results are usually part luck, so the next measurement tends to sit closer to the average whatever you do.
-
What a p-value Says
A p-value gives the probability of seeing data this extreme if there were no real effect, not the probability that the effect is real.
-
An Interval Says More
A confidence interval shows the range of values the data is compatible with, which a single estimate hides.
-
Simpson's Paradox
A trend that holds in every subgroup can reverse when the groups are combined, if the groups differ in size or composition.
-
Axes Can Lie
Truncating an axis, changing its scale or choosing the window makes the same data look dramatic or flat.
-
Find the Denominator
A count means little without knowing what it is a count out of.
-
Test Enough and Something Wins
When many comparisons are made, some will look significant by chance alone.
-
Proxies Drift
A metric stands in for something you care about, and the two separate once people optimise the metric.
-
Relative Without Absolute
A percentage change is uninterpretable without the underlying rate it changed from.