Incremental and Concurrent Marking in V8’s Orinoco GC

A 1.5 GB heap should take hundreds of milliseconds to mark, yet your traces show major GC pauses of only a few milliseconds — until one day they do not. This guide from How Mark-and-Sweep Garbage Collection Works, in JavaScript Memory Fundamentals & Runtime Mechanics, explains how V8’s Orinoco collector hides most marking work behind incremental steps and background threads, how to read --trace-gc output to see what actually paused your main thread, and what makes those techniques fail.

Symptom Root Cause Immediate Action Measurable Impact
Rare long Mark-Compact pauses on a large heap Incremental marking could not finish in time; finalisation done atomically Reduce allocation rate and heap size; check --trace-gc reasons Pauses return to single-digit ms
CPU usage higher than request work explains Concurrent marking and sweeping threads use extra cores Account for GC threads in CPU limits Correct container CPU sizing
Main thread busy with many small GC steps Incremental marking steps interleaved with JavaScript Normal; watch total, not individual steps Understand where time goes
GC pauses worse in a 1-vCPU container Background threads compete with the main thread Allocate at least 2 vCPUs to GC-heavy services Concurrent work actually runs concurrently
Latency spikes coincide with “allocation failure” reasons Heap filled before concurrent marking completed Lower allocation rate; raise heap headroom Fewer emergency full collections

Root Cause: Doing Major GC Without Stopping the World

A naive mark-and-sweep collector stops the program, traces every reachable object from the roots, then frees everything unmarked. The pause is proportional to the live heap: marking a heap of a gigabyte takes a noticeable fraction of a second, which is unacceptable for interactive pages or low-latency servers. V8’s garbage collector, known as Orinoco, attacks that pause with three techniques that together move most work off the critical path.

Incremental marking splits marking into small steps interleaved with JavaScript execution. When the old generation approaches its limit, V8 starts a marking cycle and then performs a few milliseconds of marking at a time — during allocation, in idle periods, or from scheduled tasks — rather than all at once. Concurrent marking goes further: helper threads trace the object graph in the background while JavaScript keeps running on the main thread. Parallel phases use several threads simultaneously while the main thread is paused, shortening the remaining stop-the-world work (the final marking of roots, and compaction). Sweeping — building free lists from unmarked memory — is also done concurrently.

Letting JavaScript mutate the heap while the marker is traversing it creates a correctness problem: the program could store a reference to an unmarked object into an already-scanned object, and the marker would never see it. V8 prevents this with a write barrier — a small check executed on every pointer store during marking that records the new reference for the marker, as explained in write barriers and the remembered set. The barrier is why marking can run concurrently at all, and it is also a small cost your code pays during marking.

The result is that a typical major GC shows up on the main thread as many short incremental steps plus one short finalisation pause, while the heavy lifting runs on other cores. The scheme depends on V8 starting early enough and on the program not allocating faster than marking can keep up. If the heap fills before marking completes, V8 must finish synchronously — the long pause you occasionally see — and the reason line in --trace-gc output tells you when that happened. The relationship with young-generation collection is covered in scavenger vs major GC.

Where major GC work runs in Orinoco Top: a stop-the-world collector pauses the main thread for the whole mark and sweep, about 180 milliseconds for a large heap. Bottom: Orinoco runs JavaScript on the main thread with small incremental marking steps, while two helper threads perform concurrent marking in the background. A short parallel finalisation pause of a few milliseconds ends marking, after which sweeping continues concurrently. Stop-the-world (conceptual) JavaScript mark + sweep: main thread blocked ~180 ms JavaScript Orinoco main helper 1 concurrent marking concurrent sweeping helper 2 concurrent marking yellow = incremental steps · red = short parallel finalisation pause

Step-by-Step Fix

  1. Trace GC in Node.js. Start the service with --trace-gc and reproduce the load. Each line shows the collector (Scavenge or Mark-Compact), heap size before and after, the pause duration, and a reason. Verification: you can list Mark-Compact events with their pause times.
  2. Classify major GC pauses by reason. Lines mentioning finalising incremental marking via a task or stack guard indicate the normal path; reasons such as “allocation failure” or “last resort” indicate the heap filled before concurrent marking finished. Verification: you know whether long pauses coincide with the abnormal reasons.
  3. Check CPU availability. Confirm the process has at least two usable cores (os.availableParallelism()) and that container CPU limits allow background threads to run. Verification: in a single-core environment, concurrent marking competes with JavaScript and pauses lengthen.
  4. Lower the allocation rate at peaks. Use allocation sampling to find allocators during the traffic that precedes long pauses, and reduce churn or promotion. Verification: fewer Mark-Compact events per minute and no allocation-failure reasons.
  5. Keep heap headroom. Size --max-old-space-size so the live heap sits well below the limit (for example under 70%), giving V8 time to start marking early. Verification: Mark-Compact lines show generous room between post-GC size and the limit.
  6. Re-trace and compare pause distributions. Repeat step 1 under the same load. Verification: p99 Mark-Compact pause falls to a few milliseconds and abnormal reasons disappear.
Major GC pauses before and after Before: p50 major GC main-thread pause 6 milliseconds, p99 142 milliseconds, with frequent allocation-failure reasons on a one-core container. After allocating two cores and reducing allocation at peak: p50 4 milliseconds and p99 11 milliseconds. Mark-Compact main-thread pause (ms) p50 6 before 4 after p99 142 before (1 vCPU, allocation failure) 11 after most marking still happens — it just stops blocking the main thread

Command and Code Reference

Use case: read GC traces from a Node.js service.

# One line per GC with pause time and reason; add --trace-gc-verbose for per-space detail
node --trace-gc server.js 2>&1 | grep -E "Mark-Compact|Mark-sweep" | tail -20

# Example line (format varies slightly by version):
# [4123:0x5a...]  90123 ms: Mark-Compact 612.3 (640.1) -> 401.8 (655.0) MB,
#   pooled: 0 MB, 3.21 / 0.00 ms  (+ 45.7 ms in 212 steps since start of marking,
#   biggest step 1.2 ms, walltime since start of marking 1310 ms)
#   (average mu = 0.981, current mu = 0.974) finalize incremental marking via task; GC in old space requested

Use case: measure GC pauses from inside the process. The perf_hooks GC entries report durations and kinds without restarting with flags.

// gc-pauses.js — histogram of major GC durations
const { PerformanceObserver, constants } = require('node:perf_hooks');
const majors = [];

new PerformanceObserver((list) => {
  for (const e of list.getEntries()) {
    // detail.kind distinguishes major (mark-sweep-compact) from minor (scavenge)
    if (e.detail?.kind === constants.NODE_PERFORMANCE_GC_MAJOR) majors.push(e.duration);
  }
}).observe({ entryTypes: ['gc'] });

setInterval(() => {
  if (!majors.length) return;
  majors.sort((a, b) => a - b);
  const p = (q) => majors[Math.min(majors.length - 1, Math.floor(q * majors.length))].toFixed(1);
  console.log(`major GC: n=${majors.length} p50=${p(0.5)}ms p99=${p(0.99)}ms`);
  majors.length = 0;                        // reset window; keeps memory bounded
}, 60_000).unref();

Verification and Regression Prevention

Verify with pause distributions rather than single events: under representative load, p99 major GC pause should be a few milliseconds, abnormal reasons should be rare or absent, and total GC CPU (visible in --trace-gc step totals or in process CPU accounting) should be stable. Compare before and after with the same traffic replay, because GC behaviour depends heavily on allocation rate.

Export GC pause histograms as metrics — see tracking GC pauses with perf_hooks — and alert on p99 major pause and on the share of GC time. Treat CPU limits as part of GC configuration: a deployment change that reduces a GC-heavy service to one core can quietly turn concurrent marking back into long pauses.

Major GC pause distribution under load Replaying the same traffic before and after the fix, the old configuration shows occasional tall main-thread pauses when marking cannot finish concurrently and falls back to atomic work. Afterwards pauses stay in a low band of a few milliseconds and abnormal GC reasons are absent. pause same traffic replay before: fallback to atomic pauses after: p99 a few ms

Edge Cases and Gotchas

Mutator utilisation

The mu values in trace lines estimate how much of the time the main thread spent running JavaScript rather than GC. Values close to 1.0 are healthy; a falling current mu means GC is taking a growing share, often a precursor to heap exhaustion.

Memory reducer and idle GC

When a page or process becomes idle, V8 may run additional collections to shrink the heap (the memory reducer). These show up as major GCs with no load — expected, and usually harmless.

Large heaps still cost CPU

Concurrent marking moves work off the main thread, not out of existence. A 4 GB heap still needs proportionally more marking work each cycle; it just happens on other cores. Reducing live heap size remains the most effective way to reduce total GC cost.

Browsers behave similarly

Chrome’s V8 uses the same Orinoco machinery, and the Performance panel shows the main-thread pieces as Major GC slices, as described in spotting GC pauses in a performance trace. Background marking appears on helper threads in the trace.

Frequently Asked Questions

Is V8’s garbage collector stop-the-world?

Partly. Young-generation scavenges pause the main thread briefly (with parallel helpers), and major GC has short stop-the-world phases for root marking and finalisation. Most major marking and sweeping runs incrementally or concurrently, so pauses are far shorter than the total work.

Why do I still see long major GC pauses?

Usually because the heap filled before concurrent marking finished, forcing V8 to complete the work synchronously, or because background threads could not run in a CPU-constrained container. High allocation rates and heaps near their limit make both more likely.

Does concurrent marking use more CPU?

Total GC CPU is similar or slightly higher because of write barriers and coordination, but it is spread across cores. In CPU-limited environments that extra parallelism needs real cores; otherwise it just competes with your JavaScript.

Can I tune incremental marking?

V8 exposes many internal flags, but they are not stable interfaces and rarely help. The reliable levers are application-level: allocate less, keep heap headroom, provide enough CPU, and avoid giant pointer-dense structures that take long to mark.