Skip to content
Insights
Managed Services & SRE8 min read ·

What Is an SRE Error Budget and How Do You Set One for 99.9% Uptime?

What Is an SRE Error Budget and How Do You Set One for 99.9% Uptime?

An error budget is the amount of unreliability your service is allowed to have over a defined period before you stop shipping risky changes and focus on stability. It equals 100% minus your service level objective (SLO). For a 99.9% uptime target, the error budget is 0.1%: about 43 minutes of downtime per 30-day month, or 1 failed request in every 1,000. To set one, pick a user-facing indicator, measure it over a rolling window, agree on the 99.9% target with product and engineering, and write a policy that says what happens when the budget runs out.

The rest of this post covers the details: the math, how we choose indicators, how to alert on budget consumption, and the policy that makes the number mean something.

What an Error Budget Actually Is

Most teams already argue about reliability. Product wants features shipped. Operations wants fewer changes. Without a shared number, those arguments are settled by whoever is most senior or most recently paged.

An error budget replaces that argument with arithmetic. Google's SRE book frames it this way: 100% reliability is the wrong target for almost any service, because users cannot tell the difference between 99.99% and 100%, and the cost of chasing the last fraction is enormous (Google SRE Book, "Embracing Risk"). Once you accept that some failure is acceptable, the question becomes how much. The error budget is that amount, and it is meant to be spent on releases, experiments, migrations, and planned maintenance.

SLI, SLO, SLA: Getting the Vocabulary Right

These three terms get mixed up constantly, so here is how we use them:

  • SLI (service level indicator): A measured ratio of good events to total events. Example: successful HTTP requests divided by all HTTP requests.
  • SLO (service level objective): The target for that SLI over a window. Example: 99.9% of requests succeed over a rolling 30 days.
  • SLA (service level agreement): A contractual commitment, usually with financial consequences, typically set looser than the internal SLO so you have room to react before breaching it.
  • Error budget: 1 - SLO, expressed as time or as a count of bad events.

The Math: What 99.9% Uptime Allows

A 0.1% error budget translates to the following allowances:

Window Total time Allowed downtime at 99.9%
1 day 1,440 min ~1.44 min (86.4 s)
1 week 10,080 min ~10.1 min
30 days 43,200 min 43.2 min
365 days 525,600 min ~8.76 hours

Two things stand out. First, 43 minutes a month is not much. A single bad deploy with a slow rollback can consume most of it. Second, the window matters. A 30-day rolling window is the most common choice because it is long enough to absorb a normal incident and short enough that the team still feels the consequences.

Time-Based vs. Request-Based Budgets

"Uptime" suggests a time-based budget: the service was either up or down for each minute. That model is simple but crude. A partial outage affecting 5% of users counts the same as a full outage, or doesn't count at all, depending on how you define "down."

We generally recommend request-based (event-based) budgets for request-driven services:

# Request-based error budget for a 30-day window
slo = 0.999
total_requests = 12_000_000          # observed over the window
failed_requests = 7_400              # 5xx + requests over latency threshold

budget_total = total_requests * (1 - slo)    # 12,000 allowed failures
budget_remaining = budget_total - failed_requests
pct_remaining = budget_remaining / budget_total

print(f"Allowed failures: {budget_total:,.0f}")
print(f"Remaining: {budget_remaining:,.0f} ({pct_remaining:.1%})")
# Allowed failures: 12,000
# Remaining: 4,600 (38.3%)

Request-based budgets scale naturally with traffic and capture partial degradation. Time-based budgets still make sense for batch systems, scheduled jobs, or anything without a steady request stream.

How to Set an Error Budget for 99.9% Uptime

Here is the process we follow when helping a team adopt SLOs for the first time.

1. Choose an SLI That Reflects User Experience

Measure as close to the user as practical. Load balancer logs or edge metrics are better than application-internal health checks, because they capture failures your app never sees. Common SLIs:

  • Availability: proportion of requests that return a non-5xx response.
  • Latency: proportion of requests served faster than a threshold (e.g., 300 ms at the 95th percentile is a target; "requests under 300 ms" is the SLI).
  • Freshness or correctness: for data pipelines, proportion of records processed within N minutes.

A good SLI is expressed as a ratio of good events to valid events. Here is an availability SLI in PromQL:

# Availability SLI for the checkout service over 30 days
sum(rate(http_requests_total{job="checkout", code!~"5.."}[30d]))
/
sum(rate(http_requests_total{job="checkout"}[30d]))

2. Look at Historical Performance Before Picking the Number

Do not choose 99.9% because it sounds respectable. Pull three to six months of data for your chosen SLI. If the service has historically run at 99.5%, a 99.9% SLO means your budget is exhausted on day one and the policy becomes noise. Start at a level you can meet, then tighten it deliberately.

3. Confirm 99.9% Matches What Users and the Business Need

Ask what a 43-minute monthly outage costs. For an internal reporting tool, 99.9% may be overkill. For a payment path, it might be the floor. Also check dependencies: a service cannot reliably exceed the availability of the critical components it calls in series. If your database's own target is 99.9%, your API's realistic ceiling is below that unless you design around it.

4. Pick the Window and Exclusions

Decide on a rolling 30-day window (or calendar month if it must align with SLA reporting). Document what is excluded, for example:

  • Requests from synthetic monitors or internal health checks
  • Client errors (4xx) caused by malformed requests
  • Pre-announced maintenance windows, if your stakeholders agree they should not count

Be conservative with exclusions. Every exclusion is a place where users can experience pain that your budget does not register.

5. Get Explicit Sign-Off

The SLO and its error budget must be agreed by engineering leadership and product leadership. If only the SRE team believes in the number, the policy will be overridden the first time it blocks a launch.

Writing an Error Budget Policy

The number does nothing on its own. The policy turns it into decisions. Google's SRE Workbook includes a sample error budget policy worth reading in full. A minimal version looks like this:

service: checkout-api
slo:
  sli: availability
  target: 0.999
  window: 30d_rolling
policy:
  budget_remaining_above_50pct:
    - normal release cadence
  budget_remaining_25_to_50pct:
    - release review required for high-risk changes
    - prioritize open reliability tickets
  budget_remaining_below_25pct:
    - freeze non-critical feature releases
    - security and reliability fixes only
  budget_exhausted:
    - full feature freeze until budget recovers above 10%
    - postmortem required for top contributing incidents
escalation:
  disputes: VP Engineering and VP Product jointly

What matters most is that the consequences are written down before the budget is spent, and that there is a named escalation path for disagreements.

Alerting on Burn Rate, Not Raw Errors

Alerting on every error spike creates fatigue. Alerting only when the budget is gone is too late. The better approach is burn rate: how fast you are consuming the budget relative to the rate that would exhaust it exactly at the end of the window. A burn rate of 1 uses the whole budget in 30 days. A burn rate of 14.4 uses 2% of a 30-day budget in one hour.

The SRE Workbook recommends multi-window, multi-burn-rate alerts (Google SRE Workbook, "Alerting on SLOs"). A fast-burn page for a 99.9% SLO might look like this:

- alert: CheckoutErrorBudgetFastBurn
  expr: |
    (
      sum(rate(http_requests_total{job="checkout", code=~"5.."}[1h]))
      / sum(rate(http_requests_total{job="checkout"}[1h]))
    ) > (14.4 * 0.001)
    and
    (
      sum(rate(http_requests_total{job="checkout", code=~"5.."}[5m]))
      / sum(rate(http_requests_total{job="checkout"}[5m]))
    ) > (14.4 * 0.001)
  labels:
    severity: page
  annotations:
    summary: "Checkout is burning error budget at >14x sustainable rate"

The long window (1h) confirms the problem is significant. The short window (5m) confirms it is still happening, so the alert resolves quickly once the issue is fixed. Pair this with a slower ticket-level alert (for example, burn rate 1 over three days) to catch gradual degradation.

Common Mistakes We See

  • Setting the SLO higher than the service has ever achieved. The budget is permanently negative and everyone learns to ignore it.
  • Measuring from inside the application. Crashes, network failures, and load balancer errors go uncounted.
  • No policy, or a policy nobody enforces. An error budget without consequences is just a dashboard.
  • One SLO for everything. Checkout and the admin settings page rarely deserve the same target.
  • Treating unspent budget as a success. Consistently leftover budget can mean you are moving too cautiously and could ship faster.

If you are building this practice across many services, it helps to treat SLO definitions, dashboards, and alert rules as code reviewed alongside the services themselves. That is the model we use in our managed services and SRE capabilities, and it holds up well in regulated and high-availability industries where reliability targets need to be auditable.

FAQ

What is the error budget for 99.9% uptime?

It is 0.1% of your measurement window. Over 30 days, that is 43.2 minutes of downtime, or 1 failed request per 1,000 if you use a request-based budget. Over a year, it is roughly 8.76 hours.

What is the difference between an SLO and an error budget?

The SLO is the reliability target (for example, 99.9% of requests succeed). The error budget is the inverse: the amount of failure the SLO permits (0.1%). They are two views of the same number, but the error budget is the one teams track and spend.

What happens when the error budget runs out?

Whatever your error budget policy specifies. Typically, non-critical feature releases pause and engineering effort shifts to reliability work until the budget recovers. The key is agreeing on these consequences in advance with both engineering and product leadership.

Should every service target 99.9%?

No. Set targets based on user impact, historical performance, and the reliability of dependencies. Internal tools may be fine at 99% or 99.5%, while critical customer paths may justify tighter targets. Each additional "nine" costs significantly more engineering effort.

Should SLA and SLO targets be the same?

Usually not. Keep the internal SLO stricter than the external SLA so your team gets warning, through budget burn alerts, well before a contractual breach occurs.