Resizable ArrayBuffer and Growable Memory

A binary builder — a protocol encoder, a log packer, an image assembler — grows its output by allocating a bigger ArrayBuffer, copying, and discarding the old one, and memory spikes to three times the final size while GC churns through discarded buffers. This guide from Typed Arrays, Buffers and External Memory, part of JavaScript Memory Fundamentals & Runtime Mechanics, shows how resizable ArrayBuffers, length-tracking typed arrays and ArrayBuffer.prototype.transfer() let buffers grow and shrink in place, and what memory they actually reserve.

Root Cause: Fixed-Size Buffers Force Copying #

Classic ArrayBuffers have a fixed byte length. When a builder does not know the final size in advance, the usual pattern is capacity doubling: allocate 64 KB, fill it, allocate 128 KB, copy, drop the old one, and so on. At each growth step both the old and the new buffer exist, so the peak is about the final size plus the previous capacity, and all the discarded buffers are external memory waiting for the collector — the pattern that makes large binary builders look like leaks in ArrayBuffer and Blob memory outside the JS heap. The final buffer is also usually larger than its content, and that slack is retained for as long as the result is kept.

ES2024 added resizable ArrayBuffers: new ArrayBuffer(initialLength, { maxByteLength }) creates a buffer whose byteLength can later change with buffer.resize(newLength), up to maxByteLength. Growth happens in place — no copy, no second buffer. Typed arrays created on a resizable buffer without an explicit length are length-tracking: their length follows the buffer as it grows or shrinks, so views never need to be recreated. SharedArrayBuffer has a growable counterpart with grow() (it can only grow, because other threads may be reading).

The same edition added ArrayBuffer.prototype.transfer(newLength), which moves the contents into a new buffer (optionally of a different length) and detaches the original. That gives you an explicit “trim to size and hand over ownership” step, and it can often avoid a copy when the engine can reuse the underlying memory.

What does maxByteLength cost? The specification leaves it to implementations; engines typically reserve virtual address space up to the maximum and commit physical memory only as the buffer grows, so a large maximum is cheap in physical memory but not unlimited — reservations consume address space and some engines cap the maximum. Choose a realistic upper bound rather than an astronomical one.

Step-by-Step Fix #

1. Find grow-by-copy builders. Search for code that allocates a larger ArrayBuffer/Uint8Array and copies with set() when capacity runs out. Verification: you have a list of builders and their typical final sizes.
2. Check runtime support. Resizable buffers and transfer() are available in current Chromium, Firefox, Safari and Node.js versions; feature-detect with 'resize' in ArrayBuffer.prototype and keep the old path as a fallback if you support older runtimes. Verification: detection returns true in your target environments.
3. Create a resizable buffer with a realistic maximum. new ArrayBuffer(initial, { maxByteLength: realisticMax }). Verification: buffer.resizable is true.
4. Use length-tracking views. Create new Uint8Array(buffer) or new DataView(buffer) without a length, so views follow resize(). Verification: after resize(), view.length equals the new byteLength without recreating the view.
5. Grow in place. When capacity is short, call buffer.resize(Math.min(max, newCapacity)). Verification: memory profiles show no intermediate buffers and no copy spikes.
6. Trim and hand over. When done, resize(bytesWritten) or buffer.transfer(bytesWritten) to produce an exact-size result with a single owner. Verification: the retained result’s byteLength equals its content, and the builder’s buffer is detached if transferred.

Command and Code Reference #

Use case: a binary writer that grows in place and returns an exact-size result.

// BinaryWriter using a resizable ArrayBuffer (ES2024)
export class BinaryWriter {
  constructor(initial = 64 * 1024, max = 256 * 1024 * 1024) {
    this.buffer = new ArrayBuffer(initial, { maxByteLength: max });
    this.bytes = new Uint8Array(this.buffer);   // length-tracking view
    this.view = new DataView(this.buffer);      // length-tracking view
    this.offset = 0;
  }
  #ensure(extra) {
    const needed = this.offset + extra;
    if (needed <= this.buffer.byteLength) return;
    const next = Math.min(this.buffer.maxByteLength, Math.max(needed, this.buffer.byteLength * 2));
    if (next < needed) throw new RangeError('BinaryWriter maximum exceeded');
    this.buffer.resize(next);                   // grows in place: no copy, no second buffer
  }
  u32(value) {
    this.#ensure(4);
    this.view.setUint32(this.offset, value);
    this.offset += 4;
  }
  bytesFrom(src) {
    this.#ensure(src.length);
    this.bytes.set(src, this.offset);
    this.offset += src.length;
  }
  finish() {
    // Exact-size, fixed-length result with a single owner; this writer's buffer detaches
    return this.buffer.transferToFixedLength(this.offset);
  }
}

Use case: feature-detect and fall back.

const canResize = typeof ArrayBuffer.prototype.resize === 'function';
const writer = canResize ? new BinaryWriter() : new LegacyDoublingWriter(); // same interface

Verification and Regression Prevention #

Verify with the same large build before and after: the peak external memory (in Node, process.memoryUsage().arrayBuffers; in the browser, the Task Manager footprint) should approach the final size rather than two to three times it, and the retained result should be exactly the content size. Allocation sampling should no longer show a series of growing ArrayBuffer allocations during the build.

Keep builders behind a small, tested class like BinaryWriter so the growth policy is defined once. Unit-test the maximum-size error path and the finish() detachment: after finish(), the writer’s buffer should report detached === true, which prevents accidental reuse and retention. Related transfer semantics between threads are covered in transferring ArrayBuffers vs copying between workers.

Edge Cases and Gotchas #

Views with explicit lengths do not track #

new Uint8Array(buffer, 0, 100) has a fixed length of 100 even on a resizable buffer. If the buffer shrinks below the view’s range, the view goes out of bounds and reports length 0. Use length-tracking views for builders.

Shrinking invalidates offsets #

After resize() to a smaller size, indices beyond the new length are gone. Code that cached offsets or lengths must re-read them.

Growable SharedArrayBuffers only grow #

SharedArrayBuffer instances created with maxByteLength support grow(), never shrinking, because other threads might be reading. Plan the maximum carefully for shared buffers.

Very large maxByteLength values #

Some engines limit the maximum or fail allocation when the requested reservation is too large for the address space. Use the largest size you realistically need, not a theoretical maximum.

Frequently Asked Questions #

What is a resizable ArrayBuffer? #

An ArrayBuffer created with a maxByteLength option whose byteLength can change later through resize(), up to that maximum. It lets binary data grow or shrink in place without allocating a new buffer and copying.

Does maxByteLength allocate all that memory upfront? #

Implementations typically reserve virtual address space for the maximum and commit physical memory only as the buffer grows, but the details are engine-specific. Choose a realistic maximum rather than an extremely large one.

What does ArrayBuffer.prototype.transfer do? #

It moves the buffer’s contents into a new ArrayBuffer, optionally with a different length, and detaches the original, whose byteLength becomes 0. transferToFixedLength does the same and guarantees a fixed-length result. Both express a clear ownership handover and can often avoid copying.

How much should the buffer grow each time? #

Doubling (capped at maxByteLength) keeps the number of resize calls logarithmic in the final size, which matters less than it did with copying but still reduces bookkeeping. Because in-place growth does not copy, you can also grow in fixed steps sized to your typical output without the memory penalty that fixed steps had with copying buffers. Finish with a trim to the exact length either way.

Do resizable buffers help with transferring data to workers? #

Yes, indirectly. After finish() transfers the content into an exact-size fixed-length buffer, that buffer can be passed in a postMessage transfer list, moving ownership to the worker without copying. The builder keeps no reference, so nothing on the sending side retains the data afterwards.

Can I use resizable buffers with WebAssembly memory? #

WebAssembly memory already grows via memory.grow(), and its buffer property is refreshed after growth. The JavaScript resizable buffer features are separate; see WebAssembly linear memory growth in the browser for that model.

Related #

• Typed Arrays, Buffers and External Memory — the parent topic
• Buffer.allocUnsafe and the Node.js Buffer Pool — Node’s own buffer allocation model
• Large Object Space and When Objects Skip New Space — grow-by-copy costs for regular arrays
• JavaScript Memory Fundamentals & Runtime Mechanics — the section overview