Memory-Safe Caching Patterns in JavaScript

Almost every long-running JavaScript application caches something — API responses, parsed documents, rendered fragments, computed layouts, memoised selectors — and almost every “memory leak” investigation eventually finds a cache that was never told when to forget. This topic, part of JavaScript Memory Fundamentals & Runtime Mechanics, is for frontend and Node.js engineers who want the speed of caching without its memory cost: how to bound a cache with LRU eviction, how to estimate what a cache really holds, when time-based expiry beats recency, and how to memoise without retaining every input ever seen.

Conceptual Grounding #

A cache is a deliberate retention: it keeps values reachable so that future requests are cheaper. That makes it indistinguishable from a leak in a heap snapshot — both appear as a long-lived collection retaining many objects — and the only difference is policy. A leak has no policy; a cache has a rule that decides when an entry stops being worth its memory. Every memory-safe cache answers four questions explicitly:

• What is the bound? A maximum number of entries, a maximum number of bytes, or both. Without a bound, a cache grows with the number of distinct keys it ever sees, which in real traffic is effectively unbounded.
• What is evicted first? Least recently used (LRU) keeps what is popular now; least frequently used (LFU) keeps what is popular overall; first-in-first-out is simplest; time-to-live (TTL) removes entries after a fixed age regardless of use.
• Who owns the lifetime? A module-level cache lives for the whole page or process — the same root behaviour described in module-level caches and global singleton leaks. A cache scoped to a component, request or session dies with its owner, which is often the simplest bound of all.
• Is retention strong or weak? Strong caches keep values until evicted. Weak structures — WeakMap keyed by objects, or WeakRef values — let the garbage collector decide, which avoids leaks but makes hit rates and memory unpredictable, as discussed in WeakRef deref() and object lifetime guarantees.

Two V8 details shape cache memory. First, a Map’s entries live in an internal hash table whose capacity grows in steps and does not always shrink when entries are deleted, so a cache that once held a million entries may keep a large backing table after eviction. Second, a cache entry’s real cost is its retained size — the key, the value and everything reachable only through them — which is often many times the “size” you think you are caching: a cached API response may retain the parsed object graph, the raw text and even a request object. Measuring the retained size, not the entry count, is what keeps budgets honest.

Diagnostic Workflow #

1. Inventory caches. Search for new Map(), new Set(), object literals and memoisation helpers stored at module scope or on long-lived singletons, plus third-party caches (HTTP clients, GraphQL clients, query libraries, template engines). Expected output: a list of caches with owner, key type and intended bound. Metric: number of caches with no explicit bound.
2. Measure each cache’s retained size. Take a heap snapshot after a representative long session (or soak test) and select each cache object; read its Retained Size in DevTools → Memory. Expected output: retained MB per cache. Metric: share of total heap retained by caches.
3. Check growth against traffic. Repeat the measurement after doubling the session length or request volume. Expected output: bounded caches plateau; unbounded ones grow linearly. Metric: MB per 1,000 distinct keys.
4. Record hit rates. Instrument get calls with hit and miss counters. Expected output: hit rate per cache. Metric: caches with low hit rates are candidates for removal or a smaller bound.
5. Choose and apply a policy. Bound by bytes for caches with variable-size values, by count for uniform ones; pick LRU for recency-driven workloads, TTL where freshness matters. Expected output: each cache has a documented policy.
6. Verify under load. Run a soak test and confirm each cache’s retained size stays within its budget while hit rate remains acceptable. Expected output: flat memory with stable hit rates. Metric: retained MB per cache at the end of the soak versus budget.

Code Patterns & Signatures #

Use case: the smallest correct bounded cache. A Map preserves insertion order, so deleting and re-inserting on access yields LRU order, and deleting the first key evicts the least recently used.

// lru.js — count-bounded LRU in a dozen lines
export function createLru(max) {
  const map = new Map();
  return {
    get(key) {
      if (!map.has(key)) return undefined;
      const v = map.get(key);
      map.delete(key);            // move to most-recently-used position
      map.set(key, v);
      return v;
    },
    set(key, value) {
      map.delete(key);
      map.set(key, value);
      if (map.size > max) map.delete(map.keys().next().value); // evict LRU
    },
    get size() { return map.size; },
  };
}

Use case: bound by bytes when values vary in size. A size function lets one cache hold a few large values or many small ones within the same budget.

export function createByteBoundedCache(maxBytes, sizeOf) {
  const map = new Map();          // key → { value, bytes }
  let total = 0;
  return {
    get(key) {
      const e = map.get(key);
      if (!e) return undefined;
      map.delete(key); map.set(key, e);          // LRU refresh
      return e.value;
    },
    set(key, value) {
      const bytes = sizeOf(value);
      if (bytes > maxBytes) return;              // never cache something bigger than the budget
      const old = map.get(key);
      if (old) { total -= old.bytes; map.delete(key); }
      map.set(key, { value, bytes });
      total += bytes;
      for (const [k, e] of map) {                // evict oldest until within budget
        if (total <= maxBytes) break;
        map.delete(k); total -= e.bytes;
      }
    },
    get bytes() { return total; },
  };
}

Use case: scope a cache to its owner instead of the module. When the owner goes away, the whole cache becomes garbage — no eviction logic needed.

// One cache per request (server) or per mounted view (client)
function handleRequest(req) {
  const perRequest = new Map();                  // dies with the request
  const user = (id) => perRequest.get(id) ?? perRequest.set(id, loadUser(id)).get(id);
  return renderPage(req, { user });
}

Use case: attach derived data to objects without retaining them.

// WeakMap keyed by the source object: entry disappears when the object is collected
const layoutCache = new WeakMap();
function layoutOf(node) {
  let l = layoutCache.get(node);
  if (!l) layoutCache.set(node, (l = computeLayout(node)));
  return l;
}

Symptom-to-Fix Reference #

Edge Cases & Gotchas #

Keys retain memory too #

A cache keyed by long strings — full URLs with query strings, serialised request bodies, JSON-stringified arguments — can hold as much memory in keys as in values. Hash or normalise keys, and be careful with keys built from substrings of large texts, which may be sliced strings retaining their parents.

Caching promises #

Storing the promise of an in-flight request de-duplicates concurrent callers, which is good, but a rejected promise stays cached and every caller receives the same failure. Delete the entry on rejection, and count pending promises toward the bound.

Clearing is not the same as shrinking #

map.clear() removes entries, but whether the internal table shrinks is an implementation detail. For caches that occasionally spike, replacing the Map with a new one after a clear guarantees the old table becomes garbage.

Multiple caches share one budget #

Ten caches each “bounded” at 50 MB add up to 500 MB. Keep a registry of caches with their budgets, and size budgets against the process or page memory budget as a whole, including the device tiers in setting memory budgets for low-end devices.

Library caches you did not configure #

HTTP clients, GraphQL clients, ORMs and query libraries ship with their own caches, some unbounded by default. Review their configuration — for example the cache eviction options in Apollo Client cache eviction — before blaming your own code.

Measuring and Budgeting Cache Memory #

A budget only works if it is measured the same way it is enforced. Entry counts are cheap to track but hide the variation in value sizes; byte estimates via a sizeOf function are closer to reality but still approximate, because they cannot see shared structure or engine overheads. The authoritative number is the retained size of the cache object in a heap snapshot, which includes everything that would be freed if the cache were dropped. Use snapshots to calibrate your sizeOf estimates — measure a cache with 1,000 typical entries, divide, and set the per-entry estimate accordingly — then enforce the budget at runtime with the calibrated estimate. The detailed procedure is in estimating the memory footprint of a cache.

Export each cache’s size, estimated bytes and hit rate as metrics. A dashboard that stacks cache bytes against total heap makes it obvious when caches, rather than leaks, are driving memory, and a hit-rate panel shows when a cache is no longer earning its memory. When memory pressure matters more than hit rate — mobile devices, small containers — reduce budgets first for caches whose hit rates are low or whose misses are cheap.

Finally, remember that eviction is not deletion from the heap: an evicted value is freed only when nothing else references it. If evicted entries are also stored in component state, closures or other caches, eviction frees nothing. When a cache’s measured retained size stays high after you lower its bound, follow the retainers of an evicted value to find the second owner.

Browser Caches and Server Caches Fail Differently #

The same unbounded Map behaves very differently depending on where it runs. In the browser, a cache lives as long as the page. Short visits hide the problem entirely; single-page apps that stay open all day expose it, and the growth is driven by one user’s activity — how many records they open, how many searches they run. The failure mode is gradual: interactions slow down as garbage collection works harder, and eventually a mobile browser reloads the tab. Budgets should therefore be set per device tier, and caches that exist only to speed up navigation (previously visited views, rendered fragments) are prime candidates for small LRU bounds or component-scoped lifetimes.

On a server, a module-level cache is shared by every request the process handles, so it grows with total traffic across all users — and it can leak data between users if keys are not scoped carefully. Growth is faster and less forgiving: a crawler or a traffic spike can add hundreds of thousands of distinct keys in minutes, and the process reaches its heap limit or its container memory limit and restarts. Server caches must always have a hard bound, their budget must be counted against --max-old-space-size alongside per-request memory, and anything user-specific belongs in a request-scoped or session-scoped structure rather than module scope, as discussed in request-scoped state with AsyncLocalStorage.

Both environments share one more pitfall: caching at several layers at once. An HTTP client caches responses, a data library caches normalised entities, and application code memoises derived views of the same data — three copies of similar information, each bounded independently or not at all. When memory budgets are tight, remove the redundant layers first; one well-bounded cache close to where data is consumed is usually cheaper than three generic ones.

Frequently Asked Questions #

What is the simplest way to stop a cache from leaking? #

Give it a bound and an eviction rule. A Map with LRU re-insertion on access and deletion of the first key when over the limit takes about a dozen lines and turns unbounded growth into a fixed maximum. For variable-size values, bound by estimated bytes rather than entry count.

Should I use WeakMap for caching? #

Use WeakMap when the cache key is an object whose lifetime should control the entry — derived data for DOM nodes, component instances or parsed documents. It cannot be keyed by strings or numbers and cannot be sized or iterated, so for URL- or ID-keyed caches use a bounded Map.

How big should a cache be? #

Big enough to reach the knee of the hit-rate curve for your real traffic, and no bigger. Measure hit rate at several sizes; beyond the point where it flattens, extra entries cost memory without saving meaningful work.

How do I know whether a cache is worth keeping at all? #

Measure its hit rate and the cost of a miss. A cache with a low hit rate, or one whose misses are cheap (a fast local computation), spends memory for little benefit and can often be removed or shrunk dramatically. Caches in front of slow network calls or expensive computations justify larger budgets, but still need a bound.

Can a cache make garbage collection slower even when it is bounded? #

Yes. A large, long-lived cache full of small objects adds marking work to every major collection, because the collector must trace every entry to prove it is still reachable. Bounding by bytes limits that cost too, but a cache of hundreds of thousands of tiny objects can still lengthen major GC pauses. Storing compact values — primitives, typed arrays, or serialised forms for cold entries — reduces the object count and the marking work along with the memory.

Should caches be cleared on logout or route changes? #

Anything scoped to a user must be cleared when the user changes, both for memory and for privacy. Route changes are a good moment to drop view-specific caches in single-page apps, but shared data caches (reference data, feature flags) should survive navigation. Give each cache an explicit owner and a reset function, and call the resets from one place in your session handling.

Where should cache budgets be documented? #

Next to the cache itself, in code: a short comment stating the bound, the eviction policy, the owner and the memory budget it implements, plus a link to the measurement that justified it. Keeping a small registry of all caches with their budgets makes it easy to check that the sum still fits the page or process memory budget.

Is LRU or TTL better? #

They solve different problems. LRU bounds memory by keeping what is used most recently; TTL bounds staleness by expiring entries after a fixed age. Many production caches use both: a size bound with LRU eviction plus a TTL for data that changes.

Related #

• Building a Bounded LRU Cache in JavaScript — a complete, tested implementation
• Estimating the Memory Footprint of a Cache — turning entries into megabytes
• TTL vs LRU Eviction: Memory Trade-offs — choosing an eviction strategy
• Memoization Without Unbounded Memory Growth — bounding memoised functions
• JavaScript Memory Fundamentals & Runtime Mechanics — the parent section