Buffer.allocUnsafe and the Node.js Buffer Pool

process.memoryUsage().arrayBuffers climbs into the hundreds of megabytes while your code only keeps a few thousand 32-byte keys, or a cache of message headers retains entire 64 KB socket chunks. This guide from Typed Arrays, Buffers and External Memory, part of JavaScript Memory Fundamentals & Runtime Mechanics, explains how Node.js allocates Buffers, how the shared pool and subarray() views make small buffers keep large memory alive, and when a copy is cheaper than a view.

Symptom Root Cause Immediate Action Measurable Impact
arrayBuffers far larger than the data you keep Small retained Buffers are views into 8 KB pool slabs Copy long-lived small buffers with Buffer.from(buf) or Buffer.alloc + copy Retained external memory ≈ bytes kept
Cache of message fragments retains whole chunks buf.subarray()/slice() share the parent’s memory Copy fragments before caching Socket chunks released
Sensitive data visible in freshly allocated Buffers allocUnsafe does not zero memory Use Buffer.alloc for data that may be exposed No leaking of old memory contents
Memory usage fine in heap snapshots, RSS high Buffer contents are external, outside the JS heap Monitor arrayBuffers and external Correct attribution of growth
Throughput drops when copying everything Excessive defensive copies Copy only what outlives the source Balanced speed and memory

Root Cause: Views, Pools and External Memory

A Node.js Buffer is a Uint8Array view onto an ArrayBuffer whose bytes live outside the V8 heap, as described in ArrayBuffer and Blob memory outside the JS heap. The JavaScript object is small; the memory it points to is reported separately in process.memoryUsage().arrayBuffers (and external). Because the Buffer is a view, several Buffers can point into the same underlying ArrayBuffer, and the underlying memory is freed only when every view onto it has been garbage collected.

Two mechanisms create shared memory without you asking. The first is the buffer pool. Allocating many small ArrayBuffers is expensive, so Node keeps a pre-allocated slab of Buffer.poolSize bytes (8 KB by default). Buffer.allocUnsafe(n) for small n (less than half the pool size), and conversions such as Buffer.from(string) for short strings, carve their result out of the current slab instead of allocating a new one. When the slab is used up, a new slab is created. Each small Buffer is therefore a view into an 8 KB slab, and keeping even one 16-byte Buffer from a slab keeps the entire slab alive. If your long-lived small Buffers each come from a different slab — for example, one key extracted per request over hours — you retain 8 KB per key.

The second mechanism is subarray() (and the deprecated buf.slice()), which return views sharing the parent’s memory. Network and file streams deliver data in chunks of 16–64 KB; extracting a 20-byte header with chunk.subarray(0, 20) and caching it keeps the whole chunk. This is the binary equivalent of sliced strings retaining large parent strings.

Buffer.allocUnsafe also has a security aspect unrelated to leaks: it does not zero the memory, so a buffer may contain bytes from earlier use until you overwrite it. Use Buffer.alloc (zero-filled, never pooled) whenever a buffer might be exposed before being fully written. Buffer.allocUnsafeSlow allocates unpooled, unzeroed memory for cases where you intend to retain a small buffer long-term without tying up a slab.

How small views keep big memory alive Top: an 8 kilobyte pool slab is carved into several small Buffers from allocUnsafe and Buffer.from calls. After the request, all but one 32 byte key are dropped, but that key is a view into the slab, so all 8 kilobytes stay allocated. Bottom: a 64 kilobyte socket chunk arrives and a 20 byte header is taken with subarray and cached; the cached header keeps the whole 64 kilobyte chunk alive. 8 KB pool slab (Buffer.poolSize) key 32 B dropped dropped dropped one retained 32-byte view keeps all 8 KB allocated 64 KB socket chunk hdr payload already processed — but still allocated cached header = chunk.subarray(0, 20) → the whole 64 KB stays alive fix: Buffer.from(chunk.subarray(0, 20)) copies 20 bytes into their own memory

Step-by-Step Fix

  1. Watch external memory, not only the heap. Log process.memoryUsage().arrayBuffers and external alongside heapUsed. Verification: you can see whether growth is in Buffer memory rather than JS objects.
  2. Find long-lived small Buffers. In a heap snapshot, filter by Buffer or Uint8Array and look at objects retained by caches, maps or arrays; inspect their buffer (the backing ArrayBuffer) and its byteLength. Verification: retained Buffers of a few bytes point to backing stores of 8 KB or more.
  3. Copy what outlives its source. When storing a Buffer beyond the current operation, store Buffer.from(view) (which copies), or Buffer.allocUnsafeSlow(n) + view.copy(target) for unpooled memory. Verification: stored Buffers have backing stores matching their own length.
  4. Keep views for short-lived processing. Inside a single parse or transform step, subarray() views are ideal: no copying, no allocation. Verification: views never escape into caches, closures or queues.
  5. Choose the right allocator. Use Buffer.alloc when the buffer may be exposed before being fully written, allocUnsafe for hot paths that immediately overwrite every byte, and allocUnsafeSlow for small buffers you will keep long-term. Verification: allocation sites are reviewed and documented.
  6. Re-measure under load. Run the same traffic for the same duration. Verification: arrayBuffers plateaus near the sum of retained data rather than growing with request count.
100,000 cached 32-byte keys If each cached key is a pooled view from a different request, the cache can retain up to about 800 megabytes of pool slabs. If each key is copied into its own unpooled buffer, retained external memory is about 3 megabytes plus per-object overhead. arrayBuffers retained by a 100k-key cache (worst case) pooled views up to ~800 MB (one 8 KB slab per key) copied keys ~3 MB + object overhead

Command and Code Reference

Use case: see the pool sharing directly.

// pool-demo.js — small allocations share one ArrayBuffer (the pool slab)
const a = Buffer.allocUnsafe(16);
const b = Buffer.allocUnsafe(16);
console.log(Buffer.poolSize);                   // 8192 by default
console.log(a.buffer === b.buffer);             // true: same slab
console.log(a.buffer.byteLength);               // 8192: keeping `a` keeps 8 KB

const c = Buffer.alloc(16);                     // zero-filled, never pooled
console.log(c.buffer.byteLength);               // 16

Use case: cache header bytes from stream chunks without retaining the chunks.

const headerCache = new Map();

socket.on('data', (chunk) => {
  const id = chunk.readUInt32BE(0);
  const headerView = chunk.subarray(4, 24);      // view: fine for immediate use
  processHeader(headerView);                     // short-lived use — no copy needed

  // Long-lived: copy so the 64 KB chunk can be freed
  headerCache.set(id, Buffer.from(headerView));  // copies 20 bytes into new memory
});

Use case: long-lived small buffers without tying up slabs.

// Unpooled, unzeroed allocation for buffers you will keep and fully overwrite
function detachSmall(view) {
  const copy = Buffer.allocUnsafeSlow(view.length); // own ArrayBuffer of exact size
  view.copy(copy);
  return copy;
}

Verification and Regression Prevention

A fix is verified when arrayBuffers under sustained load plateaus close to the size of the data you actually retain, and heap snapshots show retained Buffers whose backing ArrayBuffer byte lengths match their own lengths. Run the check with realistic traffic patterns — the pool effect is worst when retained buffers come from many different requests over time.

Keep arrayBuffers and external in production metrics next to heap usage, as in exposing Node.js memory metrics with prom-client, and alert on external memory growth separately. In code review, treat any subarray/slice result that is stored in a cache, map, closure or queue as needing an explicit copy.

arrayBuffers under sustained load Under sustained load, retaining small Buffers sliced from the shared pool pins each 8 KB pool chunk they came from, so arrayBuffers grows well beyond the data kept. After copying retained data into right-sized Buffers, arrayBuffers plateaus close to the size of the data actually retained. arrayBuffers sustained load retained pool slices pin 8 KB chunks Buffer.from copy for retained data

Edge Cases and Gotchas

Buffer.from(buffer) copies, Buffer.from(arrayBuffer) does not

Buffer.from(existingBuffer) copies the bytes. Buffer.from(arrayBuffer, offset, length) creates a view onto the given ArrayBuffer without copying. Read the overload carefully when writing detach helpers.

Changing Buffer.poolSize

Buffer.poolSize can be changed, and setting it to 0 effectively disables pooling for subsequent allocations. That trades allocation speed for simpler lifetimes; measure before changing it globally.

Streams may reuse or slice chunks

Some libraries hand you slices of their internal buffers. If you store data from a third-party stream, copy it; the library’s buffer may be large or reused.

Garbage collection timing

Buffer backing stores are freed only when their JavaScript view objects are collected. A small heap with little allocation may not collect for a while, so external memory can stay high even after you drop references. That is timing, not a leak; test with forced GC before concluding.

Frequently Asked Questions

What is the Node.js buffer pool?

A pre-allocated slab of memory, 8 KB by default, from which Node carves small Buffers created by Buffer.allocUnsafe and some Buffer.from conversions. It makes small allocations fast, but every small Buffer is a view into a slab, and a retained view keeps its whole slab alive.

Is Buffer.allocUnsafe dangerous?

It returns memory that is not zeroed and may contain old data, which is a security risk if the buffer can be exposed before it is fully written. It is safe and fast when you immediately overwrite every byte. Use Buffer.alloc when in doubt.

Does subarray copy data?

No. subarray() returns a view sharing the parent’s memory. It is ideal for short-lived processing but keeps the entire parent allocation alive if you store the view. Copy with Buffer.from(view) for long-term storage.

Should I disable the pool to avoid the problem?

Usually not. The pool makes small allocations fast and is harmless for buffers that die quickly, which is the common case. Copy the few small buffers that you retain long-term instead; that fixes retention without slowing down every short-lived allocation in the process.

Why doesn’t a heap snapshot show my Buffer memory?

Buffer contents live outside the V8 heap as external memory. Snapshots show the small Buffer objects and their ArrayBuffer wrappers, with byte lengths, but the bytes themselves are counted in process.memoryUsage().arrayBuffers and in RSS rather than in heap totals.