Blog
Forensic write-ups of burn-rate alert failures — the incident, the root cause, and the rule change that fixes it.
The 2-minute recording lag that cost a payments burn 130 seconds
A three-level recording cascade evaluates behind the alert reading it — replayed against a payments latency burn, the page the classifier attributes to the lag arrives 130 seconds after the ratio it depends on already had.
Read the case study →The 10x threshold that made a real search burn invisible
A fast-burn alert hardcodes a threshold ten times higher than its own 99.9% SLO math requires — the rule that should have paged never gets within reach of the number it was actually written to catch.
Read the case study →The 5-minute window that hid an 880-error burst
A single-window burn-rate alert reads the ratio over its own lookback and nothing else — replayed against a checkout error burst, the rule that should have paged never crosses threshold.
Read the case study →