Escalation policies
What happens when nobody acknowledges — page a person, then the next one.
A channel is a broadcast. An escalation policy pages a person, and pages somebody else if that person does not answer.
A policy
An ordered list of steps. Each step has targets and a timeout.
Step 1 → on-call primary (payments) wait 5 minutes
Step 2 → on-call secondary (payments) wait 5 minutes
Step 3 → engineering manager + #incidents wait 10 minutes
Step 4 → everyoneAn acknowledgement stops the escalation. Nothing else does — not a resolve by somebody else, not the alert recovering, not somebody replying in Slack.
Targets
| Target | Resolves to |
|---|---|
| A user | That person |
| A schedule | Whoever is on call now |
| A team | Everyone on that team |
| A channel | A broadcast — no acknowledgement expected |
A step whose only target is a channel cannot be acknowledged, so escalation continues past it regardless. Put the broadcast alongside a person, never instead of one.
Paging a person means paging a person
An earlier version of this system broadcast to channels when it believed it was paging a user. Every step reached the same room, nobody was individually responsible, and escalation did nothing at all.
Now a user target resolves to that user's own notification preferences — their email, their push, their phone.
Routing rules
Which alerts enter which policy is decided by routing rules, matched on severity, service, team and environment.
Team-routing rules were once silently dropped when a rule matched but the team had no policy. The alert went to the default broadcast and looked delivered. Rules now validate their targets at save time — a policy that cannot page anybody is refused rather than accepted and quietly ineffective.
Stop on resolve
When the underlying alert resolves, escalation stops. Nobody is woken at step 3 for something that recovered during step 1.
Testing a policy
Test it. A policy is a chain of assumptions — that a schedule has somebody on it, that the person's contact details are current, that the channel still exists — and each link is silently wrong until used.
Run a test after: creating a policy, a rotation change, somebody joining or leaving, and any credential rotation.
Analytics
Escalation & response analytics shows how often escalation reaches step 2 and beyond. Frequent step-2 escalation means either the primary is overloaded or the paging channel is not reaching them — both worth fixing, and both invisible without the number.
Where this behaviour lives: backend/internal/service/escalation_svc.go, backend/internal/handler/escalation_handler.go. If the code and this page disagree, the code is right — please fix the page.
Part of Incidents and on-call — Turning a reading into a page, and a page into a resolved incident.