The Sample Decides the Answer
Who was never in the data?
The idea
A result describes the group you measured, and generalises only as far as that group represents the wider one.
In the real world
A survey of current customers cannot tell you why people did not buy.
Going deeper
A result describes the group that was measured and generalises only as far as that group represents the wider one. The question of who was excluded is usually more informative than the finding.
A satisfaction survey to active users returning 92% positive is real and narrow: everyone who left is absent by construction, and so is everyone who ignores surveys. It cannot address churn at all, however large the sample. Survivorship, self-selection and non-response all operate here, and none of them are visible in the number itself.
Where it stops applying
Perfectly representative samples are rarely achievable and demanding them prevents useful work. The standard is knowing the scope of what you have, not having a flawless sample.
Why it matters
It is the first question to ask of any statistic, and it is usually unasked.
Try this today
For one finding you trust, name who was excluded from the data.
Test yourself
A satisfaction survey sent to active users returns 92% positive. What population does that describe, and which question does it not answer?
Show the answer
Only people still using the product and willing to respond. Everyone who left is absent by construction, so it cannot say anything about why people churn. The result is real and its scope is far narrower than it appears.
Learn this in the feed Answering from memory, then again days later, is what makes it stick.