TTL vs LRU Eviction: Memory Trade-offs

Your cache has a five-minute TTL, so it “cannot leak” — yet during a traffic spike it grew to 2 GB and took the service down. Or your LRU cache keeps memory flat but serves data that is hours old. This guide from Memory-Safe Caching Patterns in JavaScript, in JavaScript Memory Fundamentals & Runtime Mechanics, compares time-based and recency-based eviction by what they actually bound, shows how each behaves under load, and explains how to combine them correctly.

Symptom Root Cause Immediate Action Measurable Impact
TTL cache grows huge during traffic spikes TTL bounds age, not count: memory = key rate × TTL Add a size bound (LRU) on top of the TTL Memory capped during spikes
Expired entries still in memory Lazy expiry only removes entries when read Add periodic sweeps or evict expired entries on write Expired data freed promptly
LRU cache serves stale data LRU has no notion of age Add a TTL check on read Freshness bounded
Thousands of timers, one per entry TTL implemented with setTimeout per key Use timestamps and a single sweeper Far fewer timer objects and closures
Hit rate collapses after a burst of one-off keys Scan pollution evicts useful entries from LRU Admission filter or segmented LRU for bursty workloads Popular entries survive scans

Root Cause: The Two Policies Bound Different Things

LRU (least recently used) evicts the entry that has gone longest without being accessed whenever the cache exceeds its size limit. It bounds memory directly — the cache never holds more than max entries or maxBytes — but says nothing about age: a popular entry can live forever, even if the underlying data changed long ago.

TTL (time to live) expires each entry a fixed duration after it was written. It bounds staleness — no entry is older than the TTL — but bounds memory only indirectly: the number of live entries is roughly the rate of distinct new keys multiplied by the TTL. At 50 new keys per second and a 5-minute TTL, that is 15,000 entries; during a spike or a crawler visit at 2,000 new keys per second, it is 600,000 entries. The same TTL that was safe at normal traffic becomes a memory bomb, with the growth appearing exactly when the service is already under load. That is why a TTL alone does not make a cache memory-safe, a point that applies with extra force on servers, as in caching vs memory bloat in SSR data layers.

Implementation details matter too. Lazy expiry checks the timestamp on read and deletes stale entries then; it is cheap but lets expired entries that are never read again sit in memory indefinitely. Active expiry removes expired entries proactively — either with a periodic sweep or by scanning a few entries on each write. Implementing TTL with one setTimeout per entry is the worst of both: it allocates a timer and a closure per key, those timers keep the entries (and their closures’ contexts) alive until they fire, and in Node.js thousands of active timers add their own overhead.

The robust design is both: a size bound with LRU eviction to cap memory, plus a TTL checked lazily on read and enforced actively by a single sweeper to cap staleness and free expired entries promptly.

Entries during a traffic spike Normal traffic creates 50 new keys per second; a ten-minute spike creates 2,000 per second. A TTL-only cache with a five-minute TTL rises from about 15,000 entries to about 600,000 during the spike. An LRU-only cache and an LRU plus TTL cache both stay at their maximum of 20,000 entries. After the spike, the TTL-only cache drains only as entries expire. 600k 0 traffic spike (2,000 new keys/s) TTL only (5 min) LRU (max 20k) ± TTL time

Step-by-Step Fix

  1. Classify the cache’s requirement. Decide whether the priority is freshness (prices, permissions, feature flags), memory (large values, many keys), or both. Verification: the requirement is documented next to the cache.
  2. Estimate worst-case key rate. From logs, find the peak rate of distinct new keys (including bots and scans). Multiply by the TTL. Verification: you know the worst-case entry count a TTL-only cache would reach.
  3. Always add a size bound. Put an LRU max or maxBytes on every TTL cache, sized from your memory budget. Verification: a replay of peak traffic keeps the cache at or below its bound.
  4. Check TTL on read. Store a write timestamp with each entry and treat entries older than the TTL as misses (deleting them). Verification: no read returns data older than the TTL.
  5. Sweep expired entries actively. Run one periodic sweeper (for example every 30 seconds) that deletes expired entries from the front of the insertion-ordered map, instead of per-entry timers. Verification: expired, never-read entries disappear within one sweep interval; active timer count is constant.
  6. Protect against scan pollution if needed. For workloads with bursts of one-off keys, admit an entry only on its second request (a small “seen once” filter) or use a segmented LRU. Verification: hit rate for popular keys stays stable during bursts.
Per-entry timers versus one sweeper With one setTimeout per entry, one hundred thousand cached entries create one hundred thousand timers and closures, adding roughly 15 to 20 megabytes and keeping entries alive until their timers fire. With timestamps and a single sweeper, there is one timer and one closure in total, and expired entries are removed on each sweep. setTimeout per entry 100,000 timers 100,000 closures + contexts ~15–20 MB extra entries pinned until their timer fires timestamp + single sweeper 1 timer 1 closure 8 bytes of timestamp per entry expired entries removed each sweep

Command and Code Reference

Use case: an LRU cache with TTL, lazy expiry on read and one sweeper.

// lru-ttl.js
export class LruTtlCache {
  #map = new Map();                         // key → { value, at }
  constructor({ max = 10_000, ttlMs = 60_000, sweepMs = 30_000 } = {}) {
    this.max = max;
    this.ttlMs = ttlMs;
    // One timer for the whole cache; unref() so it never keeps Node alive
    this.sweeper = setInterval(() => this.sweep(), sweepMs);
    this.sweeper.unref?.();
  }
  get(key, now = Date.now()) {
    const e = this.#map.get(key);
    if (!e) return undefined;
    if (now - e.at > this.ttlMs) { this.#map.delete(key); return undefined; } // lazy expiry
    this.#map.delete(key); this.#map.set(key, e);                            // LRU refresh
    return e.value;
  }
  set(key, value, now = Date.now()) {
    this.#map.delete(key);
    this.#map.set(key, { value, at: now });
    while (this.#map.size > this.max) this.#map.delete(this.#map.keys().next().value); // size bound
  }
  sweep(now = Date.now()) {
    // Entries are roughly ordered by last write/refresh; stop at the first fresh one
    for (const [key, e] of this.#map) {
      if (now - e.at <= this.ttlMs) break;
      this.#map.delete(key);
    }
  }
  close() { clearInterval(this.sweeper); this.#map.clear(); }
}

Use case: a “seen twice” admission filter against scan pollution.

// Admit keys to the main cache only on their second request within a window
const seenOnce = new LruTtlCache({ max: 50_000, ttlMs: 60_000 }); // tiny values: just `true`
function shouldAdmit(key) {
  if (seenOnce.get(key)) return true;
  seenOnce.set(key, true);
  return false;
}

Verification and Regression Prevention

Replay a traffic spike — or a synthetic burst of distinct keys — against the cache and confirm that entry count and estimated bytes never exceed the size bound, while no read returns entries older than the TTL. Check that the active timer count stays constant as the cache fills (in Node, process.getActiveResourcesInfo() should show one timer for the sweeper, not thousands).

Keep a test that advances time with fake timers, writes entries, and asserts that sweep() removes expired ones and that get() never returns them. In production, export entry count, evictions by reason (size versus expiry) and hit rate; a sudden rise in size-based evictions during normal traffic means the bound is too small, while a rise in expiry evictions with low hit rate may mean the TTL is too short. For memoised functions specifically, see memoization without unbounded memory growth.

Entry count during a burst of distinct keys Replaying a burst of distinct keys, a TTL-only cache grows with every new key until entries begin to expire, so its peak depends on traffic. A cache with both a TTL and a size bound stays capped at the bound throughout while still never returning entries older than the TTL. entries burst of distinct keys TTL only: peak follows traffic TTL + size bound Active timers should stay constant: one sweep timer, not one per entry.

Edge Cases and Gotchas

Refresh-on-read changes TTL semantics

The implementation above refreshes recency on read but keeps the original write time, so TTL measures age since write. If you want “expire after N minutes of inactivity” instead, update at on read — but then popular stale data never expires.

Sweep order assumptions

The early break in sweep() relies on entries being roughly ordered by time. Because reads move entries to the end without changing at, a few expired entries may sit behind fresh ones until they are read or pushed out by the size bound. That is acceptable for memory safety; use a full scan if strict prompt removal matters.

Clock changes

Date.now() can jump when the system clock changes. For TTLs, performance.now() (monotonic) is safer in long-running processes.

Negative caching

Caching “not found” results avoids repeated expensive misses but can multiply keys during scans of non-existent IDs. Give negative entries a shorter TTL and include them in the size bound.

Frequently Asked Questions

Does a TTL prevent a cache from leaking memory?

Not reliably. A TTL limits how long entries live, so the number of entries equals roughly the rate of new keys multiplied by the TTL. Under traffic spikes or scans that can be enormous. Add a size bound to cap memory.

When should I use TTL instead of LRU?

Use TTL when stale data is unacceptable beyond a known age — permissions, prices, configuration. Use LRU when memory is the constraint. Most production caches need both: a size bound for memory and a TTL for freshness.

Why not use setTimeout to expire each entry?

Per-entry timers allocate a timer and a closure per key, keep entries alive until they fire, and scale poorly to many thousands of keys. Store timestamps, expire lazily on read, and run one periodic sweeper instead.

What is scan pollution?

A burst of one-off requests — a crawler, a batch job, a user paging through many items once — fills an LRU cache with entries that will never be reused, evicting popular ones. Admission filters or segmented LRU variants protect the frequently used entries.