Sizing Worker Pools with Piscina

You moved CPU-heavy work — image processing, PDF generation, parsing — into a Piscina worker pool, and now the service uses three times the memory it used to, the task queue grows during spikes, and occasionally a worker dies with an out-of-memory error. This guide from Worker Threads Memory Isolation, part of Node.js Server-Side Memory Management, explains the memory cost of each worker, how to size a pool from CPU and memory budgets together, and how Piscina’s options bound queues and per-worker heaps.

Symptom Root Cause Immediate Action Measurable Impact
Baseline RSS jumps after adding a pool Each worker has its own isolate, heap and loaded modules Size maxThreads from memory as well as CPU Predictable baseline
Memory spikes when traffic bursts Unbounded task queue holds every pending task’s payload Set maxQueue; reject or shed load when full Bounded queued memory
Worker OOM crashes on large inputs Per-worker heap too small, or no per-task memory limits Set resourceLimits; split large tasks Controlled failures instead of process instability
Worker memory grows over thousands of tasks Module-level caches or leaks inside worker code Recycle workers, fix worker-side caches Flat per-worker memory
Payload copying doubles memory Large buffers structured-cloned into workers Transfer ArrayBuffers instead of copying One copy per payload

Root Cause: Workers Multiply Everything a Process Loads

A Node.js Worker runs a separate V8 isolate on its own thread. It has its own heap (young and old generation), its own compiled code, and its own copy of every module the worker script imports — the isolation model described in worker threads memory isolation. An idle worker that imports a few libraries commonly costs from about 10 MB to several tens of megabytes of RSS before it does any work; a worker that loads a large dependency such as a PDF engine or an image library costs much more. A pool of 16 workers multiplies that baseline by 16.

Piscina manages a pool of such workers: it spawns between minThreads and maxThreads, queues tasks when all workers are busy, and passes task payloads with postMessage. Four memory components follow. The baseline is threads × per-worker idle memory. The working set is threads × the peak memory of one task, because every busy worker holds its current task’s data. The queue holds pending tasks with their payloads; without maxQueue, a burst of large tasks can queue unbounded memory in the main thread. And payload transfer: payloads are structured-cloned by default, so a 50 MB input exists in both threads unless you transfer its ArrayBuffer, as explained in transferring ArrayBuffers vs copying between workers.

CPU sets the useful upper bound on threads — more CPU-bound workers than cores add memory without adding throughput — and memory sets the other bound: threads × (baseline + peak task) + queue must fit in the container alongside the main thread. Per-worker heap limits (resourceLimits, detailed in capping worker heap with resourceLimits) turn an oversized task into a contained worker failure instead of a process-wide problem.

Budgeting a worker pool A 4 gigabyte container holds the main thread at 400 megabytes, eight workers with a 60 megabyte baseline each for 480 megabytes, eight concurrent task peaks of 250 megabytes each for 2 gigabytes, and a queue capped at 64 tasks of about 5 megabytes for 320 megabytes, totalling about 3.2 gigabytes and leaving headroom. With sixteen workers the task peaks alone would need 4 gigabytes. 4 GB container, 8 workers ≈ 3.2 GB main 8 × baseline 8 × task peak (250 MB) queue Same container, 16 workers 16 × task peak ≈ 4 GB — exceeds the limit under full load

Step-by-Step Fix

  1. Measure one worker. Start a pool with one thread, record RSS idle, then run a representative large task and record the peak. Verification: you have per-worker baseline and peak-task memory figures.
  2. Compute the budget. Container limit − main thread peak − safety margin = memory for workers and queue. Verification: you know how many MB the pool may use.
  3. Choose maxThreads. Take the smaller of CPU cores available for this work and (worker budget − queue budget) ÷ (baseline + peak task). Verification: the pool size fits both CPU and memory.
  4. Cap the queue. Set maxQueue so queued payloads fit their budget, and handle the rejection when it is full (return 503, retry later). Verification: a burst test shows queue length at the cap and memory bounded.
  5. Set per-worker heap limits and recycle. Configure resourceLimits.maxOldGenerationSizeMb for workers and, for libraries that fragment memory, recycle workers periodically. Verification: oversized tasks fail in the worker without destabilising the process; long runs show flat per-worker memory.
  6. Transfer large inputs and outputs. Pass ArrayBuffers in the transfer list (Piscina supports Piscina.move() for this). Verification: RSS during large tasks no longer shows double copies.
Queue memory during a 10× burst During a burst of ten times normal traffic, an unbounded queue grows to 1,800 pending tasks and main-thread memory rises by about 9 gigabytes before the container is killed. With maxQueue set to 64, the queue stays at 64, excess requests receive 503 responses, and memory rises by about 320 megabytes. high 0 OOMKilled unbounded queue maxQueue 64 + 503 on overflow burst duration

Command and Code Reference

Use case: a pool sized from measured budgets.

// pool.js
const { Piscina } = require('piscina');
const os = require('node:os');

const WORKER_BASELINE_MB = 60;      // measured: idle worker RSS
const TASK_PEAK_MB = 250;           // measured: largest representative task
const WORKER_BUDGET_MB = 2600;      // container limit - main thread - margin
const QUEUE_TASKS = 64;             // × ~5 MB payload ≈ 320 MB

const byMemory = Math.floor(WORKER_BUDGET_MB / (WORKER_BASELINE_MB + TASK_PEAK_MB));
const byCpu = Math.max(1, os.availableParallelism() - 1);   // leave a core for the main thread

const pool = new Piscina({
  filename: require.resolve('./render-worker.js'),
  maxThreads: Math.min(byMemory, byCpu),
  minThreads: 2,
  maxQueue: QUEUE_TASKS,
  idleTimeout: 60_000,                                     // shrink when idle
  resourceLimits: { maxOldGenerationSizeMb: 384 },         // contain oversized tasks
});

module.exports = pool;

Use case: shed load when the queue is full and transfer payloads.

const { Piscina } = require('piscina');
const pool = require('./pool');

app.post('/render', async (req, res) => {
  const input = await readBodyAsArrayBuffer(req);
  try {
    // Piscina.move marks the ArrayBuffer for transfer instead of structured clone
    const out = await pool.run({ input: Piscina.move(input) });
    res.type('application/pdf').send(Buffer.from(out));
  } catch (err) {
    if (err.message.includes('Task queue is at limit')) return res.sendStatus(503);
    throw err;
  }
});

Verification and Regression Prevention

Task path through a bounded pool A request arrives. If the queue is full it gets a 503 response. Otherwise the task waits in the queue and runs in a worker with a heap limit. A normal task returns its result. A task that exceeds the heap limit rejects, and Piscina replaces the worker while the main process keeps serving. request queue < maxQueue? bounded payloads worker resourceLimits heap result returned task rejects (OOM) worker replaced 503, retry later

Verify under three load shapes: steady load (RSS plateaus at the computed budget), bursts (queue stays at its cap, overflow requests get fast 503s, memory bounded), and a long soak with thousands of tasks (per-worker memory flat, or recycled before it grows). Track pool metrics — utilisation, queue size, wait time, task duration — alongside RSS; Piscina exposes several of these directly.

Recompute the budget whenever task types, dependencies or container sizes change, and keep the measured constants next to the pool configuration with the date they were measured. When workers need hard memory caps, combine resourceLimits with retry or fallback logic so a single oversized input does not repeatedly crash workers.

Edge Cases and Gotchas

Container CPU limits versus availableParallelism

os.availableParallelism() may report host cores rather than the container’s CPU quota in some environments. Size from the configured quota when it is lower, or CPU-bound workers will contend and add memory without throughput.

Native libraries in workers

Image and PDF libraries often have native allocations and caches per worker, which are outside resourceLimits. Measure RSS per worker, not just heap, and configure library caches inside worker scripts.

minThreads keeps memory resident

Workers kept alive by minThreads hold their baseline even when idle. Lower it for services with long idle periods; idleTimeout shrinks the pool above the minimum.

Recycling costs warm-up

Terminating and replacing workers releases their accumulated memory but pays the startup and JIT warm-up again. Recycle after a number of tasks or when memory crosses a threshold, not on every task.

Frequently Asked Questions

How much memory does a Node.js worker thread use?

At least its own V8 heap and compiled code plus every module it imports — commonly 10 MB to several tens of megabytes idle, and much more for heavy dependencies. Measure your worker script’s idle RSS and peak task memory rather than relying on a general figure.

How many threads should a Piscina pool have?

The smaller of the CPU cores available for the work and the number of workers whose baseline plus peak task memory fits your memory budget. More CPU-bound workers than cores add memory without improving throughput.

Why does maxQueue matter for memory?

Every queued task holds its payload in the main thread. Without a limit, bursts can queue thousands of payloads and exhaust memory. A capped queue with fast rejection keeps memory bounded and gives callers a clear signal to retry.

Should large task inputs be transferred or copied?

Transfer them when the main thread no longer needs the data: moving an ArrayBuffer avoids a second copy in the worker. Piscina’s Piscina.move() wraps a buffer for transfer.

What happens when a worker exceeds its resourceLimits?

The worker is terminated with an out-of-memory error, and the task’s promise rejects. The main process continues, and Piscina replaces the worker. Handle the rejection by retrying with a smaller input or failing the request.

Can workers share memory to reduce duplication?

Read-mostly data can be shared through SharedArrayBuffer, avoiding a copy per worker, as described in the SharedArrayBuffer guide in this topic. Module code and JavaScript objects cannot be shared between isolates.

Should each HTTP request create its own worker?

No. Starting a worker costs its full baseline plus startup and JIT warm-up time, so per-request workers multiply memory with concurrency and add latency. A long-lived pool amortises that cost across many tasks and gives you one place to cap threads and queue length.