Design for Failure

Not if it fails — what happens when it does?

The idea

Components fail, so a resilient system is one where a failure is contained rather than one where nothing fails.

In the real world

A payment provider outage degrades checkout to a queue instead of an error page.

Going deeper

Every dependency has downtime, so the real choice is between a considered degradation and whatever the code happens to do — which is usually an error page and lost orders.

An architecture review answering will the payment provider be down with it will not has not designed the behaviour that will eventually be needed. Deciding it in advance is cheap: queue the order and confirm later, fall back to a second provider, or fail with a clear message and a saved basket. All three are better than the default, and the default is what you ship by not choosing.

Where it stops applying

Designing for every failure mode is not economical. Rank dependencies by what their absence costs and how likely it is, and accept a plain failure for the low end.

Why it matters

It changes the goal from preventing every failure, which is impossible, to deciding how each one behaves.

Try this today

Pick one external dependency and decide what your system does when it is unavailable.

Test yourself

An architecture review asks how the system behaves if the payment provider is down for an hour. The answer is that it will not be. What has not been designed?

Show the answer

The behaviour during the outage that will eventually happen. Every dependency has downtime, so the real choice is between a considered degradation and whatever the code happens to do, which is usually an error page and lost orders.

Learn this in the feed Answering from memory, then again days later, is what makes it stick.

More in Product & Systems Thinking

Where to Push a System Queues Reveal Capacity Bottlenecks Govern Flow Defaults Are Decisions Observability Enables Learning Constraints Shape Design

All Product & Systems Thinking lessons