Estimating the Memory Footprint of a Cache

You need to set a byte budget for a cache, but nobody knows whether 20,000 entries is 5 MB or 500 MB, and JSON.stringify(value).length gives numbers that do not match what the heap shows. This guide from Memory-Safe Caching Patterns in JavaScript, part of JavaScript Memory Fundamentals & Runtime Mechanics, shows how to measure what a cache really retains, how to turn that into a cheap per-entry estimator, and which hidden costs to include.

Symptom Root Cause Immediate Action Measurable Impact
Byte budget enforced, yet heap grows past it sizeOf underestimates real object cost Calibrate sizeOf against snapshot retained size Enforced budget matches real memory
Entries “small” but cache retains huge memory Values reference large graphs (responses, DOM, closures) Inspect retainers of a sample entry Finds the hidden reference to cut
JSON length used as size estimate Serialised size ≠ heap size (headers, pointers, shapes) Measure per-entry heap cost directly Estimates within ~20%
Keys consume unexpected memory Long string keys (URLs, serialised args) Include key size in the estimate or hash keys Honest totals; smaller keys
Numbers differ between Chrome and Node Pointer compression and engine versions change object sizes Calibrate in the runtime you ship Correct budget per runtime

Root Cause: Logical Size Is Not Heap Size

A cache entry’s memory cost is the retained size of everything that would become garbage if the entry were removed: the key, the value object, all objects reachable only through it, and the entry’s slot in the Map’s hash table. That can differ from intuitive measures by an order of magnitude in either direction.

Serialised length underestimates small objects: every JavaScript object has a header (hidden class pointer, properties and elements pointers) of a few dozen bytes before any fields, every property value that is an object or a non-small number is a pointer to another heap object, and short strings carry a header too. A record that serialises to 60 bytes of JSON can easily cost 150–250 bytes on the heap across its objects. The per-field cost also depends on the runtime — pointer compression halves pointer sizes in Chrome compared with typical Node builds.

Serialised length can also overestimate — when many entries share the same substructures (the same nested config object, interned property-name strings), the shared part is counted once on the heap but once per entry in JSON. And it can miss things entirely: a cached value that holds a reference to a response object, a DOM node or a closure retains all of that, while its JSON form shows only the data. External memory — ArrayBuffers, ImageBitmaps, decoded audio — does not appear in JSON at all and only partially in heap totals.

The reliable approach is to measure, then calibrate: fill a cache with a representative sample, measure the retained size with a heap snapshot or a heap delta after forced GC, divide by the number of entries, and use that per-entry figure (or a formula fitted to value characteristics) as your runtime sizeOf. Check once in a while that the calibration still holds, especially after data-shape changes or runtime upgrades. Snapshot mechanics for retained size are covered in retained size vs shallow size explained.

What one cache entry really costs A single entry's retained size is broken into parts: Map table slot about 24 bytes, key string about 80 bytes for a URL, value object header and fields about 56 bytes, nested arrays and strings about 400 bytes, and an accidentally retained full response object of about 18 kilobytes. JSON length of the value was only 380 bytes. Retained size of one entry (approximate) Map table slot ~24 B key (URL string) ~80 B value header + fields ~56 B nested arrays, strings ~400 B kept response object ~18 KB — the real cost, invisible in JSON JSON.stringify(value).length = 380 — off by ~50× for this entry

Step-by-Step Fix

  1. Build a representative sample. Collect a few thousand real entries (from production-like data), or reproduce the workload that fills the cache. Verification: the sample covers typical and large values.
  2. Measure retained size with a snapshot. Fill the cache with N sample entries, take a heap snapshot in DevTools → Memory (or via node --inspect), and read the cache object’s Retained Size. Verification: you have MB for N entries.
  3. Cross-check with a heap delta. In Node with --expose-gc, record heapUsed after GC, insert N entries, collect again and diff; include arrayBuffers for external data. Verification: delta and snapshot agree within about 10–20%.
  4. Inspect an outlier entry. Select a large entry in the snapshot and follow its retainers and containment. Verification: you know whether its size comes from the data itself or from an accidental reference (response, DOM, closure).
  5. Cut accidental references, then calibrate. Store compact projections of the data, then derive a sizeOf formula — for example a base cost plus a per-item cost for arrays and a per-character cost for strings — fitted to the measurements. Verification: sum(sizeOf) over the sample is within about 20% of the measured retained size.
  6. Enforce and re-check. Use the calibrated sizeOf for the byte bound, and repeat the measurement after data-shape or runtime changes. Verification: the cache’s measured retained size stays near its enforced bytes figure.
Estimators versus measured retained size For five thousand sample entries the measured retained size is 3.1 megabytes. JSON length estimates 1.9 megabytes, a 39 percent underestimate. A fixed 64 bytes per entry estimates 0.3 megabytes, badly low. A calibrated formula of base cost plus per-array-item and per-character costs estimates 2.9 megabytes, within 7 percent. 5,000 entries: estimate vs measured (MB) measured (snapshot) 3.1 JSON length 1.9 (−39%) fixed 64 B/entry 0.3 (−90%) calibrated formula 2.9 (−7%)

Command and Code Reference

Use case: measure per-entry heap cost in Node.js.

// measure-cache.mjs — node --expose-gc measure-cache.mjs
const tick = () => new Promise((r) => setTimeout(r, 0));
async function settled() {
  for (let i = 0; i < 2; i++) { await tick(); globalThis.gc(); }
  const m = process.memoryUsage();
  return m.heapUsed + m.arrayBuffers;             // include external buffers
}

const sample = loadSampleValues();                // array of representative values
const cache = new Map();

const before = await settled();
sample.forEach((v, i) => cache.set(`key:${i}`, v));
const after = await settled();

const perEntry = (after - before) / sample.length;
console.log(`~${perEntry.toFixed(0)} bytes per entry for ${sample.length} entries`);

Use case: a calibrated estimator. Fit the constants to your measurements; the structure matters more than the exact numbers.

// Calibrated against snapshots of real entries (update when data shape changes)
const BASE = 96;            // object header, Map slot, fixed fields
const PER_ITEM = 40;        // each element of the `items` array incl. its object
const PER_CHAR = 1.1;       // Latin-1 strings ≈ 1 byte/char + headers amortised

export function sizeOfReport(r) {
  let chars = r.title.length + r.id.length;
  for (const it of r.items) chars += it.label.length;
  return BASE + r.items.length * PER_ITEM + Math.ceil(chars * PER_CHAR);
}

Use case: store a compact projection instead of the raw response.

// Before: caching the whole response retains headers, raw body, and more
cache.set(url, response);

// After: keep only what the UI reads
const data = await response.json();
cache.set(url, { id: data.id, title: data.title, items: data.items.map(({ id, label }) => ({ id, label })) });

Verification and Regression Prevention

Your estimate is good when the enforced byte total and the measured retained size agree within a margin you can live with (20% is typical), across both typical and large values. Re-run the measurement script in CI against a fixed sample: if a schema change makes entries heavier, the per-entry figure changes and the test alerts you to recalibrate.

Keep calibration constants next to the estimator with a comment stating when and how they were measured and in which runtime. Export the cache’s estimated bytes as a metric and occasionally compare it with snapshot retained size in staging. When the gap widens, look first for new accidental references — the most common reason a cache suddenly costs more than its estimate — using the techniques in reading the Retainers panel.

Estimated versus measured bytes As the cache fills with a fixed sample of typical and large values, the enforced byte estimate and the measured retained size rise together and stay within about twenty percent. If a schema change makes entries heavier, the measured line pulls away and the CI measurement flags the need to recalibrate. bytes entries in the cache measured retained size enforced estimate (within ~20%) Run the measurement script in CI against a fixed sample of values.

Edge Cases and Gotchas

Shared substructures

If entries share nested objects (for example a common author object), the heap stores them once, so per-entry measurements depend on sample composition. Calibrate with realistic sharing, or estimate shared parts separately.

Two-byte strings

Strings with characters outside Latin-1 use two bytes per character. If your data mixes languages, calibrate on realistic text or use a per-character factor that reflects the mix.

External memory

Values containing ArrayBuffers, Blobs or ImageBitmaps cost memory outside the JS heap. Include arrayBuffers in Node deltas and use byte lengths directly in sizeOf for such values.

Snapshot overhead in large samples

Very large samples make snapshots slow. A few thousand entries usually suffice for calibration; scale linearly and confirm with a larger soak test.

Frequently Asked Questions

How do I find out how much memory a JavaScript object uses?

Take a heap snapshot and read the object’s retained size, which includes everything reachable only through it. For many similar objects, measure the heap delta after forced garbage collection and divide by the count. There is no built-in sizeof operator.

Is JSON.stringify length a good estimate of memory?

Not on its own. It ignores object headers and pointers, double-counts shared structures and misses references that are not serialised. Use it only as an input to a formula calibrated against real measurements.

Should cache keys count toward the budget?

Yes, when they are large. Long URLs, serialised arguments or composite string keys can be a significant share of memory. Include them in sizeOf, or hash them to shorter keys.

Why do my measurements differ between Chrome and Node?

Object sizes depend on V8 build options, especially pointer compression, which Chrome enables and typical Node builds do not. Calibrate in the runtime where the cache actually runs.