Caching vs Memory Bloat in SSR Data Layers

Your server-rendering process caches fetched data to cut response latency, but heapUsed climbs request after request until the process is killed or restarts under load — a direct extension of the SSR heap exhaustion pattern covered in Node.js Server-Side Memory Management.

Root Cause #

An SSR data layer typically memoizes expensive fetches (database rows, upstream API responses, rendered fragment HTML) in a module-level Map so the next request for the same key skips the round trip. That Map lives for the lifetime of the Node.js process, which is exactly the problem: nothing ever removes an entry unless you write code to do so. Each new distinct key — a new product ID, a new query string, a new locale — adds a permanent resident object to the heap. This is a textbook case of reference counting vs tracing GC doing exactly what it’s supposed to: the tracing garbage collector correctly refuses to collect anything still reachable from a live root, and the cache Map is reachable for as long as the process runs, so every value inside it is “live” by definition even when it will never be read again.

The failure compounds under real traffic for two separate reasons. First, cache key cardinality is usually higher than developers assume — pagination parameters, sort orders, and locale variants multiply a handful of logical pages into thousands of cache keys, each holding a full rendered payload. Second, teams frequently conflate two very different lifetimes: data that should live across many requests (a product catalogue entry) and data that is specific to one request (a signed-in user’s session, request headers, personalization flags). When the latter gets written into the same shared, process-wide cache, the cache grows in direct proportion to concurrent unique users rather than to distinct content, and it never shrinks because no request “owns” the entry it wrote.

The fix has three independent levers, and all three are usually needed together: bound the cache by size (LRU eviction), bound it by time (TTL expiry), and separate scope (per-request objects never enter the shared, long-lived cache). The diagram below shows the difference between the unbounded pattern and a correctly bounded one.

Step-by-Step Fix #

1. Capture a baseline. Run your SSR server under representative load, then take a heap snapshot: DevTools → Memory → Heap Snapshot → Comparison view (attach via node --inspect and open chrome://inspect). Search the snapshot’s “Class filter” for Map and note the retained size of your cache instance. Expected Output / Verification: The snapshot shows one or more Map objects whose retained size climbs snapshot-over-snapshot under repeated requests, confirming the cache — not the request objects — is the growth source.

2. Identify the cache module. Grep the codebase for module-level new Map() or plain object literals used as memo stores in your data-fetching layer (getServerSideProps, loader functions, resolver caches). Expected Output / Verification: You can point to the exact file and line where the cache is declared, and confirm it has no delete, .clear(), or size check anywhere in the codebase.

3. Replace it with a bounded LRU cache. Implement (or install) an LRU structure with a fixed maxSize and TTL support, then swap all reads/writes to go through it. See the Command & Code Reference below for the implementation. Expected Output / Verification: cache.size never exceeds maxSize even under sustained load; confirm with a log line or a /debug/cache-stats endpoint.

4. Move per-request state out of the shared cache. Audit every write into the cache for values derived from req.headers, session tokens, or per-user personalization, and relocate them to a request-scoped object created fresh per request and discarded when the response finishes. Expected Output / Verification: Grepping the cache’s key namespace no longer shows session IDs or user IDs as key prefixes.

5. Re-run the load test and compare snapshots. Repeat step 1 after the fix: DevTools → Memory → Heap Snapshot → Comparison view, filtering by the LRU cache’s constructor name. Expected Output / Verification: Retained size plateaus near maxSize × average-entry-size and does not increase further with additional unique requests.

6. Add a runtime memory guard. Start the process with node --max-old-space-size=2048 server.js so a regression fails fast with an out-of-memory crash and a stack trace instead of silently degrading response times. Expected Output / Verification: Under an intentionally unbounded test build, the process throws FATAL ERROR: Reached heap limit well before it would have affected production traffic silently.

Command & Code Reference #

The unbounded pattern most SSR data layers start with — a plain Map used as a permanent memo store:

// Anti-pattern: unbounded module-level cache.
// Every distinct key stays resident forever.
const cache = new Map();

async function getProductData(id) {
  // Return cached value if present — but
  // nothing ever removes an entry below.
  if (cache.has(id)) return cache.get(id);
  const data = await db.fetchProduct(id);
  cache.set(id, data); // grows without bound
  return data;
}

A bounded replacement using LRU eviction plus TTL expiry, sized from a memory budget rather than a guessed entry count:

// Bounded cache: LRU eviction + TTL expiry.
// maxSize caps entry count; TTL caps age.
class BoundedCache {
  constructor(maxSize = 500, ttlMs = 60_000) {
    this.maxSize = maxSize;
    this.ttlMs = ttlMs;
    this.map = new Map(); // insertion-ordered
  }

  get(key) {
    const entry = this.map.get(key);
    if (!entry) return undefined;
    if (Date.now() > entry.expiresAt) {
      this.map.delete(key); // TTL expired
      return undefined;
    }
    // Re-insert to mark as most-recently used
    this.map.delete(key);
    this.map.set(key, entry);
    return entry.value;
  }

  set(key, value) {
    if (this.map.size >= this.maxSize) {
      // Map preserves insertion order —
      // first key is the least-recently used
      const oldestKey = this.map.keys().next().value;
      this.map.delete(oldestKey); // evict LRU
    }
    this.map.set(key, {
      value,
      expiresAt: Date.now() + this.ttlMs,
    });
  }
}

const cache = new BoundedCache(500, 60_000);

async function getProductData(id) {
  const hit = cache.get(id);
  if (hit) return hit; // fast path, no fetch
  const data = await db.fetchProduct(id);
  cache.set(id, data); // bounded write
  return data;
}

Per-request scoping keeps session-derived data out of the shared cache entirely, so it cannot accumulate across concurrent users:

// Per-request scope: created fresh per request,
// discarded automatically when it goes out of
// scope — never written into the shared cache.
function handleRequest(req, res) {
  // Local to this call only — not module-level
  const requestContext = {
    userId: req.session.userId,
    locale: req.headers['accept-language'],
  };
  // Pass requestContext explicitly instead of
  // stashing it in the shared `cache` Map above
  return renderPage(requestContext, cache);
}

Verification & Regression Prevention #

Set explicit metric targets rather than relying on “it looks stable”: cap the shared cache’s retained size at a fixed budget (for example 100 MB, verified via DevTools → Memory → Heap Snapshot → Comparison view filtering on your cache class), and alert if process.memoryUsage().heapUsed grows more than 10% between two samples taken 30 minutes apart under steady traffic. Add a lightweight monitoring endpoint that exposes cache.map.size alongside maxSize, and page on-call if size sits at maxSize continuously for over 15 minutes — that indicates eviction pressure high enough to hurt hit rate, not just a memory risk.

For regression prevention, add a CI check that fails the build if a new module-level Map or object literal is introduced as a cache without going through the shared BoundedCache helper — an ESLint rule restricting direct new Map() usage outside an allow-listed cache module works well here. Pair it with a load-test assertion in your pipeline: run N requests with unique keys exceeding maxSize, then assert cache.map.size <= maxSize and that heap growth between the first and last snapshot stays within a fixed tolerance (for example under 5 MB).

Frequently Asked Questions #

Does clearing require.cache also free my SSR data cache? #

No. Clearing require.cache only removes cached module exports so a file can be re-required; it has no effect on a Map or LRU cache you built yourself for fetched data. That cache is a live object graph rooted in a module-level variable and is only freed when you explicitly evict entries or the process restarts.

Should I use a WeakRef cache instead of an LRU cache? #

Use WeakRef only when something else in your application already holds a strong reference to the same object and you just want a non-owning lookup. WeakRef entries can be collected at any time the garbage collector runs, so they are unsuitable as your primary SSR cache — pair a bounded LRU cache for guaranteed hits with WeakRef only for opportunistic secondary lookups.

How big should I set maxSize for an LRU response cache? #

Start from a memory budget, not an entry count: measure the average retained size of one cached value in a heap snapshot, decide how much heap you can dedicate to the cache (for example 100 MB), then divide to get maxSize. Re-measure after traffic grows, since average entry size drifts as your data model changes.

Related #

• SSR Heap Exhaustion & Per-Request Memory — the parent guide covering the broader per-request memory lifecycle this page bounds.
• Fixing Memory Leaks in Next.js Server Rendering — framework-specific SSR leak patterns beyond caching.
• Reference Counting vs Tracing GC Algorithms — why reachable cache entries are never collected automatically.
• Node.js Server-Side Memory Management — the main section for server-side memory diagnostics and fixes.
• Diagnosing Node.js Memory with heapdump & Clinic.js — tooling for confirming cache growth in production.