Tuning --max-semi-space-size for Throughput

An allocation-heavy Node.js service — JSON APIs, SSR, data transformation — spends 10–20% of its CPU in scavenges, --trace-gc shows a Scavenge every few milliseconds under load, and old space fills with objects that should have died young. This guide from Memory Limits and Out-of-Heap Errors in Node.js, part of JavaScript Memory Fundamentals & Runtime Mechanics, explains how the young generation is sized, how --max-semi-space-size changes GC behaviour, and how to measure whether a larger value actually helps your service.

Symptom Root Cause Immediate Action Measurable Impact
Scavenges every few ms under load Young generation small relative to allocation rate Benchmark --max-semi-space-size=32/64/128 Scavenge frequency drops proportionally
Old space grows quickly, then major GCs reclaim a lot Short-lived objects promoted because they were alive at scavenge time Larger semi-space gives objects time to die young Less promotion, fewer major GCs
GC share of CPU 10%+ in profiles Frequent scavenges with high survival Increase semi-space; reduce allocation GC CPU falls, throughput rises
Memory footprint rose after tuning Young generation reserves up to ~3× the semi-space size Account for it in container limits No surprise OOMKills
No improvement after raising the value Allocation is not the bottleneck, or survivors are genuinely long-lived Revert; optimise elsewhere Avoids paying memory for nothing

Root Cause: Scavenge Frequency Is Allocation Rate Divided by Young-Generation Size

V8 allocates new objects in the young generation, which is organised as semi-spaces. New objects are bump-allocated into one semi-space; when it fills, a scavenge copies the survivors into the other semi-space (or promotes objects that already survived once into old space) and the roles swap. The cost of a scavenge is proportional to the number of surviving objects, not to the garbage, which is what makes it cheap — as long as most objects are dead when it runs. The mechanics are covered in scavenger vs major GC.

The semi-space size sets how often scavenges happen: a service allocating 400 MB/s with 16 MB semi-spaces triggers a scavenge roughly every 40 ms; with 64 MB, roughly every 160 ms. That matters in two ways. First, fixed overhead per scavenge — stopping JavaScript, scanning roots and the remembered set, synchronising helper threads — is paid more often with small semi-spaces. Second, and usually more important, survival depends on timing. An object that is alive for 30 ms — say, the parsed body and intermediate objects of an in-flight request — survives a scavenge that happens every 10 ms and may be promoted into old space, where it becomes garbage that only a major GC can reclaim. With scavenges every 160 ms, the same object is dead by the time the scavenger looks, and costs nothing.

V8 grows the semi-space dynamically up to a maximum, which you set with --max-semi-space-size (in MB). Defaults depend on the V8 version and system memory and are relatively small (commonly around 16 MB per semi-space on 64-bit systems), tuned for a wide range of workloads including memory-constrained ones. For throughput-oriented servers with ample memory, larger values such as 32, 64 or 128 MB often reduce GC CPU and promotion substantially. The cost is memory: the young generation reserves space for two semi-spaces plus related areas, so its footprint is roughly three times the semi-space size, and larger semi-spaces can make each scavenge pause longer if survival is high.

Why larger semi-spaces reduce promotion Top timeline with 16 megabyte semi-spaces: scavenges happen about every 40 milliseconds. A request's temporary objects created just before a scavenge are still alive at the scavenge and survive, then get promoted at the next one. Bottom timeline with 64 megabyte semi-spaces: scavenges happen about every 160 milliseconds, so the same 30 millisecond request objects are already dead when the scavenger runs and cost nothing. 16 MB semi-space: scavenge every ~40 ms at 400 MB/s request objects alive at a scavenge → copied, soon promoted 64 MB semi-space: scavenge every ~160 ms request objects dead long before the next scavenge → free scavenge

Step-by-Step Fix

  1. Measure the baseline under realistic load. Replay production-like traffic and record throughput, p99 latency, GC time (via --trace-gc or perf_hooks GC entries), scavenge count per second and old-space growth rate. Verification: you have numbers to compare against, gathered with the same load generator and duration.
  2. Estimate allocation rate. From --trace-gc, count scavenges per second and multiply by the young-generation size, or record an allocation sampling profile. Verification: you know roughly how many MB per second the service allocates.
  3. Try larger values in steps. Restart with --max-semi-space-size=32, then 64, then 128, repeating the same load test each time. Verification: scavenge frequency falls roughly in proportion; record GC time, promotion and latency for each.
  4. Watch promotion and major GCs. Compare old-space growth rate and Mark-Compact frequency between runs. Verification: larger semi-spaces reduce promotion if short-lived objects were being promoted prematurely.
  5. Check the memory cost. Compare RSS between runs; the young generation can take about three times the semi-space size. Verification: the chosen value fits within your container memory limit together with --max-old-space-size.
  6. Pick the smallest value that captures most of the gain. Choose the point where further increases give little improvement. Verification: the production rollout shows the same reduction in GC time and no memory-limit incidents.
GC share and throughput by semi-space size For an allocation-heavy JSON API, GC takes 17 percent of CPU at 16 megabytes, 10 percent at 32, 6 percent at 64 and 5 percent at 128. Throughput rises from 4,100 to 4,650, 4,900 and 4,950 requests per second respectively, so most of the gain arrives by 64 megabytes while the young generation's memory cost keeps growing. GC share of CPU by --max-semi-space-size (illustrative) 16 MB 17% GC · 4,100 req/s 32 MB 10% GC · 4,650 req/s 64 MB 6% GC · 4,900 req/s 128 MB 5% GC · 4,950 req/s · young gen ~384 MB diminishing returns past 64 MB; memory cost keeps rising

Command and Code Reference

Use case: run the same load test at several semi-space sizes.

#!/usr/bin/env bash
# semi-space-bench.sh — compare GC share and throughput per setting
for size in 16 32 64 128; do
  node --max-semi-space-size=$size --trace-gc server.js > gc-$size.log 2>&1 &
  PID=$!
  sleep 3                                               # let the server start
  npx autocannon -c 100 -d 60 http://localhost:3000/api/report > load-$size.txt
  kill $PID
  echo "== $size MB: scavenges=$(grep -c Scavenge gc-$size.log) \
marks=$(grep -c Mark-Compact gc-$size.log)"
  grep -E "Req/Sec|Latency" load-$size.txt | head -4
done

Use case: measure GC time share from inside the process. Useful in production canaries where restarting with --trace-gc is inconvenient.

// gc-share.js — fraction of wall time spent in GC, per minute
const { PerformanceObserver } = require('node:perf_hooks');
let gcMs = 0;
new PerformanceObserver((list) => {
  for (const e of list.getEntries()) gcMs += e.duration;   // all GC kinds
}).observe({ entryTypes: ['gc'] });

setInterval(() => {
  console.log(`GC share: ${((gcMs / 60_000) * 100).toFixed(1)}%`);
  gcMs = 0;
}, 60_000).unref();

Use case: set the flag without changing the start command.

# NODE_OPTIONS applies to every node process started in this environment
export NODE_OPTIONS="--max-semi-space-size=64 --max-old-space-size=1536"
node server.js

Verification and Regression Prevention

A tuning change is justified when the load test shows lower GC share, fewer Mark-Compact events and equal or better latency, and production metrics after rollout show the same trend without memory-limit incidents. Record the chosen value and the benchmark results next to the deployment configuration, so the next person knows why it is there and can re-run the benchmark after major Node upgrades — defaults and heuristics change between V8 versions.

Export GC metrics continuously (see tracking GC pauses with perf_hooks) so a later code change that increases allocation rate shows up as rising GC share even with the tuned setting. Tuning buys headroom; it does not replace reducing allocation in hot paths.

Evidence that justifies a semi-space change A change to --max-semi-space-size is justified when the load test shows a lower share of time in GC, fewer Mark-Compact events, and equal or better latency, and production metrics after rollout show the same trend without memory-limit incidents. Record the value and results next to the deployment configuration. Same load test, before vs after GC share Lower share of CPU time spent in GC. Mark-Compact events Fewer, because less is promoted early. Latency Equal or better at p50 and p99; no memory-limit incidents.

Edge Cases and Gotchas

Memory-constrained containers

In a 512 MB container, a 128 MB semi-space can consume most of the memory by itself. Size the young generation and --max-old-space-size together, and leave room for native memory and buffers, as discussed in how Node.js sizes the default heap in containers.

High survival makes scavenges longer

If most young objects genuinely survive — a service that builds large long-lived caches at high rate — larger semi-spaces make each scavenge copy more, increasing pause length. The tuning helps only when most allocations die young.

Worker threads

Workers get their own young generations. Flags on the main process apply to workers started from it, and resourceLimits.maxYoungGenerationSizeMb in the Worker constructor lets you set it per worker.

Latency-sensitive services

Fewer but larger scavenges can slightly increase individual pause times. For strict latency budgets, look at p99 and p999, not just throughput, when choosing the value.

Frequently Asked Questions

What does --max-semi-space-size do?

It sets the maximum size, in megabytes, of each of the young generation’s semi-spaces. Larger semi-spaces mean scavenges happen less often, so more short-lived objects die before being examined, reducing GC work and premature promotion to old space.

What value should I use?

There is no universal answer. Benchmark your service at 16, 32, 64 and 128 MB under realistic load and pick the smallest value that captures most of the improvement in GC share and latency, within your memory budget. Many allocation-heavy servers settle between 32 and 64 MB.

Does it affect the heap limit?

The young generation’s size adds to the total heap in addition to --max-old-space-size. Plan for roughly three times the semi-space size of extra memory when you increase it.

How do I know whether objects are being promoted prematurely?

Compare old-space growth with what you expect to be long-lived. In --trace-gc output, watch how much old space grows between Mark-Compact events under steady load, and how much each Mark-Compact reclaims. If major collections regularly reclaim most of what was promoted, those objects were short-lived and only survived because scavenges happened too often — the case a larger semi-space addresses.

Is this useful in browsers?

Browsers manage V8 flags themselves, and pages cannot change them. The same principle applies in the browser, though: reducing allocation rate in hot code has the same effect as a larger young generation, because it spaces scavenges further apart.