Setting a Heap Size Budget in Your CI Pipeline

The gate machinery in automated memory leak detection in CI is straightforward; the number it compares against is where teams get stuck, and a badly chosen number is worse than no gate at all because it trains people to ignore red builds. This page, part of Browser DevTools & Performance Profiling Workflows, covers calibrating that number from real data and keeping it honest over time.

Symptom Root Cause Immediate Action
Budget fails builds that added a feature Budget expressed as absolute heap size Re-express it as bytes retained per interaction
Gate is green while the app gets steadily worse Budget set far above the measured figure Ratchet it down to mean plus three sigma
Nobody knows why the number is 120 KB Value has no recorded calibration Commit the date, build and method beside it
Budget raised repeatedly with no discussion Value stored in a CI variable, not the repo Move it into the repository so changes are reviewed
Figure differs between local and CI Different browser version or viewport Pin both and recalibrate once

Root Cause: A Budget Is a Statistical Claim

Every budget encodes a claim: this measurement is stable enough that a value above X means something changed in the application rather than in the environment. That claim is testable, and testing it is the whole calibration process.

Start with linearity. Run the same flow at ten, twenty and fifty iterations. If the per-iteration figure is 41, 40 and 41 KB, the measurement is linear and a budget on it is meaningful. If it is 120, 70 and 45 KB, you are measuring a fixed warm-up cost divided by a growing denominator, and no budget on that figure will behave — the fix is another warm-up pass, not a bigger number.

Then measure the spread. Run one build five times without changing anything. A pinned browser on a dedicated runner typically produces a spread of two to four per cent; a shared runner with parallel jobs can double that. The budget is the mean plus three times the spread, which makes a false failure rare without making the gate blind — at three sigma, roughly one run in three hundred fails spuriously, which is tolerable when the alternative is missing real regressions.

Finally, record the calibration. The number, the date, the build it came from, and the method. Six months later this is the difference between a team that can reason about the budget and a team that treats it as folklore.

Calibrating from the spread, not from a guess Five measurements cluster between thirty-nine and forty-three kilobytes with a mean of forty-one. One standard deviation is one point six kilobytes, so the budget is placed at mean plus three sigma, just under forty-six, rounded up to fifty for memorability. Five runs of one build — the budget comes out of this picture, not out of the air 36 KB 41 KB 46 KB 50 KB mean 41.0 KB mean + 3σ = 45.8 KB budget 50 KB σ = 1.6 KB · rounding up to 50 costs almost nothing and makes the number memorable

Step-by-Step Fix

  1. Check linearity at three iteration counts. Command: run the gate with ITERATIONS=10, 20, 50. Verification: the per-iteration figures agree within a few per cent.
  2. Run the same build five times. Verification: you have five numbers and can compute a mean and a spread; if the spread exceeds 10%, fix the environment before setting any budget.
  3. Compute mean plus three sigma and round up. Verification: the rounded value is above every observed run and below any figure you would consider a real regression.
  4. Commit it beside the test with a comment. Verification: git blame on the budget line shows who set it, when, and why.
  5. Add a lower-it-when-fixed rule to the review checklist. Verification: the budget’s history shows it moving down over time, not staying flat for a year.

Command & Code Reference

A calibration script that produces the number rather than asking you to invent it.

// Run the gate repeatedly against the SAME build and report the statistics
// that determine the budget.
async function calibrate(runGate, runs = 5) {
  const values = [];
  for (let i = 0; i < runs; i++) {
    const { perIteration } = await runGate();
    values.push(perIteration / 1024);          // work in KB
    console.log(`run ${i + 1}: ${values[i].toFixed(1)} KB`);
  }
  const mean = values.reduce((a, b) => a + b) / values.length;
  const variance = values.reduce((a, v) => a + (v - mean) ** 2, 0) / values.length;
  const sigma = Math.sqrt(variance);
  const budget = Math.ceil((mean + 3 * sigma) / 10) * 10;   // round up to 10 KB
  console.log(`mean ${mean.toFixed(1)} KB · σ ${sigma.toFixed(1)} KB → budget ${budget} KB`);
  return budget;
}

The committed budget, in the repository, with the provenance that makes it reviewable.

// memory-budgets.js — reviewed like any other code.
// Calibrated with scripts/calibrate.js; lower this whenever a fix lands.
module.exports = {
  // flow                        budget KB   calibrated   build
  'dashboard→reports→dashboard':      50,  // 2026-07-31   #4821
  'open+close detail panel':          20,  // 2026-07-31   #4821
  'infinite scroll, 200 rows':       340,  // 2026-06-12   #4590
};

The assertion, written so a failure message contains everything needed to act.

const budgets = require('./memory-budgets');

function assertWithinBudget(flow, perIterationBytes) {
  const budgetKb = budgets[flow];
  const actualKb = perIterationBytes / 1024;
  if (actualKb <= budgetKb) {
    console.log(`PASS ${flow}: ${actualKb.toFixed(1)} KB (budget ${budgetKb} KB)`);
    return;
  }
  // Include the ratio: 1.1× is a nudge, 8× is a broken teardown.
  const ratio = (actualKb / budgetKb).toFixed(1);
  throw new Error(
    `FAIL ${flow}: ${actualKb.toFixed(1)} KB is ${ratio}× the ${budgetKb} KB budget. ` +
    `See the attached heap snapshot for the retaining constructor.`
  );
}
Fixed budget versus a ratchet Under a fixed budget the measured figure drifts up from forty to a hundred and ten kilobytes over twelve commits without ever failing. Under a ratcheting budget the same drift triggers a failure at commit seven, because each earlier fix lowered the ceiling behind it. The same twelve commits under two policies 0 70 KB 140 KB commits → fixed budget 140 KB — never fires measured figure drifts 40 → 110 KB ratcheting budget first failure — caught at commit 7 The drift is identical in both cases; only one policy notices it while it is still one commit's worth of work.

Verification & Regression Prevention

Prove the budget is meaningful by breaking it deliberately: revert a known teardown fix on a branch and confirm the gate fails with a ratio well above one. Then confirm it does not fire spuriously by running the current build ten times; zero failures at three sigma is the expected result, and one failure means the spread was underestimated and the environment needs attention before the number does.

Keep the measured value as a time series in the same place your bundle-size numbers live, and review it in the same rhythm. The failure mode a budget cannot catch on its own is slow drift under the ceiling, and the only defence against that is a chart someone looks at plus the habit of lowering the number whenever the measurement improves.

Five properties of a budget people will trust A budget should be per interaction rather than absolute, calibrated from measured spread, committed in the repository, annotated with its calibration date and build, and lowered whenever a fix lands. Each row names the failure that property prevents. If any row is missing, the gate gets disabled within a quarter Per interaction, not absolute prevents: failing every build that legitimately adds a feature Derived from measured spread prevents: flaky failures that teach people to re-run the job Committed in the repository prevents: silent raises with no reviewer and no record Annotated with date and build prevents: a number nobody can defend or safely change Ratcheted down after every fix — prevents: slow drift under a stale ceiling

FAQ

Should the budget be an absolute heap size or a per-interaction figure?

Per interaction, always. Absolute heap size drifts upward with every legitimate feature, so an absolute budget fails builds that added functionality rather than builds that leaked. A per-interaction figure is unaffected by baseline growth and isolates retention, which is the thing you actually want to gate on.

What is a reasonable starting budget?

There is no universal number — it depends entirely on what the interaction does. Calibrate from your own clean build rather than borrowing a figure. As a sanity check, a route transition in a typical application settles somewhere between 20 and 150 KB per cycle; a figure in megabytes usually means the flow legitimately caches something on first use.

What if the budget has to be raised for a legitimate feature?

Raise it, in a pull request, with the reason in the commit message. The value of the budget is not that it never moves; it is that moving it requires a reviewer to agree the increase is intentional. A budget stored in a CI variable can be raised by anyone at 2am with no record.