Review every alert rule change without reviewing every alert rule
You own the monitoring platform, not the services on it. BurnRateSurgeon puts the review in CI: a pull request that touches a burn-rate rule is replayed against real history before it merges.
A rule review is one person reading YAML for a service they do not operate. It scales with the number of teams shipping, and it is the first thing dropped in a week that has an incident in it.
Every team is on a slightly different stack
One group is on Prometheus, one migrated to Mimir, the acquired team is on Datadog. A check that understands one of them is a check the rest of the fleet skips.
A generated ruleset is still an unreviewed ruleset
Sloth and Pyrra turn an SLO into rules and validate the YAML they just wrote. Nothing in that chain replays one hour of the team's own history against the result.
Three things the platform gets once, for everybody
A gate on the pull request
The gate replays the changed rules against the window before the merge and reports the verdict on the diff — which incidents the new rule would have missed, and which storms it would have started.
It runs from a committed action bundle, so the version gating your repository is a file in your repository and is reviewed the way the rest of your CI is.
The same deterministic replay engine reads Prometheus, Mimir, Grafana Cloud and VictoriaMetrics rule groups and Datadog composite monitors, and produces the same classified report for each.
Cross-vendor is the constraint the product is built under rather than an integration list: a signature catalogued on one stack stays readable from the others.
The failure-signature corpus is served over MCP as well as over HTTP, so an agent reviewing a rule change asks what a failure class is and gets the answer your engineers get.
Only entries whose owners consented to publication are served, and the public read is anonymous — the same gate decides what an agent sees and what a search engine indexes.
Every incident, and the rule that should have caught it
The replay grades your rules against the incidents that really happened in the window. Each miss keeps its evidence — which incident, and the reason the alert stayed silent.
An example replay report: five incidents over seven days, two of them missed by CheckoutErrorBudgetBurnFast, each miss listed with the incident it was attributed from.
2 of these 5 incidents burned budget without paging anyone. Every one keeps the rule it was graded against.
Prescriptive
A catalogue of failures, each with the rule change that ends it
Every classified miss resolves to a signature in the public corpus: the root cause in one paragraph, and the diff that fixes it. Readable without an account, indexed for search.
The public failure-signature corpus: three catalogued entries, each with its root-cause class, its identifier and the vendors it applies to.
burnratesurgeon.com/corpus
26 public entries · no account needed
Window boundary miss
sig_synth_checkout_burst_between_evaluations
A 90-second wave of 503s from the checkout service burned 4% of the monthly error budget, but the page never fired: the burst sat between two evaluations of the 1h long window and was averaged away before the rule next ran.
A single node stayed NotReady for three hours after a kernel upgrade. Because the inhibition rule matched only on `cluster`, that one node alert suppressed every burn-rate page in the cluster, including a real recommendations outage on healthy nodes.
Applies to: Prometheus
Cardinality explosion hides the signal
sig_synth_tenant_label_explodes_ratio_series
The burn-rate rule kept the `tenant` label, so onboarding a batch of trial accounts multiplied the recorded series by forty. The rule group hit the per-user series limit, the recording rule was refused, and the alert evaluated against a series that had stopped updating.
Applies to: Mimir, Grafana Cloud
Live
The number that says the fix held
Shadow-mode keeps replaying after the rule ships, so the dashboard is a measurement and not a launch claim. One figure: critical alerts your rules predicted correctly, week by week.
An example north-star dashboard: critical alerts correctly predicted per week over eight weeks, above the counters for the last thirty days.
burnratesurgeon.com/dashboard/checkout-slo
Critical alerts correctly predicted
17
Week of 2026-08-24 · 65 alerts over 8 weeks
24
Replays run
19 graded against alert history
41
Predicted correctly
Alerts the rules raised that really fired
2
Rules about to fail
Alerts that fired and the rules would have missed
6
Storms predicted
Alerts the rules would have raised and reality did not
CheckoutErrorB…
PaymentsLatenc…
5%
Alert burn
△Warning
2 of 43 paging-worthy incidents went unpaged in the last 30 days.
Last 30 days
We already run promtool and a rules linter in CI.
Both check that the rule is well-formed. Neither runs it. A rule that passes every linter can still sit between two evaluations of its own long window — that failure is invisible to a file that parses, and it is the one the replay finds.
Validate your burn-rate rules against real history. The open-source CLI is free forever — the SaaS adds shadow-mode, PR gates, and a cross-vendor failure-signature corpus.