Node.js Worker Threads and Child Processes

Parallel Work Choices

Worker threads run JavaScript in parallel inside one process. Child processes provide stronger runtime and memory isolation or execute external programs. Both need bounded pools, message contracts, cancellation, and shutdown ownership.

Choose by Workload and Isolation

Keep asynchronous I/O on the event loop. Use worker threads for sustained CPU-heavy JavaScript such as parsing, image transforms, or calculations when measurement shows event-loop harm. Use child processes for external executables, separate Node runtimes, different privileges, or stronger crash and memory isolation.

Creating a worker or process has startup, memory, serialization, and scheduling cost. Do not create one for every tiny request. Use a bounded pool and queue with admission limits, deadlines, and overload behavior.

I/O is not automatically a worker task.
Measure task cost before parallelizing.
Bound workers, queued jobs, and per-job memory.

Worker Thread Message Contract

Workers have separate JavaScript heaps and communicate with structured-clone messages, transferable ArrayBuffers, MessagePorts, or carefully synchronized SharedArrayBuffer data. Define message version, job ID, input limits, result shape, and error serialization.

Transferring an ArrayBuffer moves ownership and detaches it from the sender. Shared memory avoids copies but introduces data races; use Atomics and a reviewed protocol. Most application work is safer with immutable messages.

Run One CPU Task in a Worker

// square-worker.mjs
import { parentPort } from 'node:worker_threads';
parentPort.on('message', ({ id, value }) => {
  parentPort.postMessage({ id, result: value * value });
});

// caller.mjs
import { Worker } from 'node:worker_threads';
const worker = new Worker(new URL('./square-worker.mjs', import.meta.url));
worker.postMessage({ id: 1, value: 12 });
worker.once('message', console.log);

Output

{ id: 1, result: 144 }

A real service should reuse a bounded pool and correlate every response with its job ID.

Worker Failure and Pool Ownership

Handle messageerror, error, and exit. Reject the current job when a worker exits unexpectedly, replace capacity according to policy, and avoid retrying non-idempotent work automatically. A thrown worker error does not cross as the original application error object.

A pool assigns at most one incompatible task per worker, tracks queue depth, cancels expired jobs cooperatively, and terminates stuck workers after a deadline. During shutdown, stop accepting jobs, drain bounded work, then terminate remaining workers.

spawn, execFile, exec, and fork

spawn starts an executable with streaming stdin, stdout, and stderr and suits long output. execFile starts a file directly and buffers output for a convenient callback or promise. exec runs through a shell and should not receive untrusted interpolation. fork starts another Node.js module with an IPC channel.

Synchronous variants block the event loop and belong only in controlled startup or small CLI contexts. Buffered exec and execFile output has a maximum; use spawn for potentially large output and consume both stdout and stderr to avoid stalls.

Safe External Program Execution

Use a fixed executable and validated argument array, a minimal environment, explicit cwd, timeout, AbortSignal, and output byte limits. Decide whether termination targets only the child or a process tree. Platform quoting and signal behavior differ, which is another reason to avoid shell strings.

Child success means the process exited with the expected code and required output was valid. Handle spawn errors, nonzero exit, signal termination, malformed output, stderr policy, and timeout separately.

Spawn with Arguments and Bounded Ownership

import { spawn } from 'node:child_process';

const child = spawn(process.execPath, ['--version'], {
  stdio: ['ignore', 'pipe', 'pipe'],
  windowsHide: true,
  timeout: 2000
});
let output = '';
child.stdout.setEncoding('utf8');
child.stdout.on('data', chunk => { output += chunk; });
child.on('close', code => console.log(code, output.trim()));

The executable and arguments are separate, output is consumed, and the process has a deadline.

Cluster and Multiple Instances

The cluster module can run Node processes that share a server port, but many deployments scale independent process or container instances behind a load balancer. Choose one ownership model for restarts, health, logs, and capacity rather than layering several supervisors blindly.

State needed by all instances belongs in an external store or a deliberate routing strategy. In-memory sessions, rate counters, queues, and caches become inconsistent when traffic moves between processes.

Parallelism Verification

Benchmark end-to-end latency, throughput, event-loop delay, CPU, memory, serialization cost, pool queue time, and cancellation. More workers than available CPU can increase contention and tail latency.

Test malformed messages, worker crash, task timeout, output overflow, child refusal, nonzero exit, signal shutdown, queue saturation, and repeated pool replacement. Ensure no orphan process survives application termination.

Before you move on

Parallel Work Check

4 checks

I choose workers for CPU work and processes for executable or isolation needs.
I use bounded reusable capacity.
I avoid shell interpolation and consume child output.
I test crashes, deadlines, saturation, and shutdown.

Node JS Questions Learners Ask

Usually not. Node already handles asynchronous I/O without blocking the JavaScript thread; workers are most useful for measured CPU-heavy JavaScript.

execFile launches an executable directly without a shell by default, reducing shell injection risk. Arguments still require domain validation.

Previous Next