JavaScript runs in a single-threaded environment with an event loop, an architecture that is very easy to reason about. It’s a continuous loop executing incoming work. Said work can schedule more of it for the future.

while (hasWorkToDo) {
    /* Run timers, I/O callbacks,
       check for incoming connections,
       and so on... */
    doWork();
}

Synchronous work runs immediately; asynchronous work runs when there is no synchronous work to left to perform (or simply, “later”). Ideally, the execution profile of the your application should allow the event loop to run frequently in order to perform background work (e.g. accepting new connections, running timers, etc).

  
gantt
    title Ideal Execution Profile
    dateFormat HH
    axisFormat %H
    todayMarker off
    section Event Loop
      Background Work    :milestone, crit, 01, sameline
      doWork()              :01, 02
      Background Work    :milestone, crit, 02, sameline
      doWork()              :02, 03
      Background Work    :milestone, crit, 03, sameline
      doWork()              :03, 04

This design implies that performing synchronous work is a Big Deal: for every continuous moment it runs, the event loop cannot perform any work – none!

/* setImmediate registers a
   callback on the event loop */
setImmediate(() => {
   console.log("This will at some point in the future");
});

/* synchronous work that you 
   will never see the end of */
findNthPrime(9999999);
  
gantt
    title Blocking Event Loop For a Long Time
    dateFormat HH
    axisFormat %H
    todayMarker off
    section User Code
      setImmediate()        :01, 03
      findNthPrime(9999999) :03, 10
      console.log(...)      :10, 12
    section Event Loop
      Background Work    :milestone, crit, 01, sameline
      doWork()              :01, 10
      Background Work    :milestone, crit, 10, sameline
      doWork()              :10, 12

In server contexts, one such request can block all others indefinitely.

/* If a request to /computePrimes
   has been sent, this route will
   never respond. It will timeout. */
app.get("/home", () => {
    return response("Welcome Home!");
});

app.get("/computePrimes", () => {
    /* synchronous work that you 
       will never see the end of. */
    return response(findNthPrime(9999999));
});
  
gantt
    title One Request Blocking Other Requests
    dateFormat HH
    axisFormat %H
    todayMarker off
    section Client
      GET /home             :01, 03
      GET /computePrimes    :crit, 02, 10
      GET /home             :active, 04, 12
      GET /home             :05, 14
    section User Code
      handle /home          :01, 03
      handle /computePrimes :crit, 03, 10
      findNthPrime(9999999) :crit, 04, 09
      handle /home          :active, 10, 12
      handle /home          :12, 14
    section Event Loop
      Background Work    :milestone, crit, 01, sameline
      doWork()              :01, 03
      Background Work    :milestone, crit, 03, sameline
      doWork()              :crit, 03, 10
      Background Work    :milestone, crit, 10, sameline
      doWork()              :active, 10, 12
      Background Work    :milestone, crit, 12, sameline
      doWork()              :12, 14

There are three solutions to these scenarios.

  1. Throw more nodes at it
  2. Refactoring findNthPrime to perform work asynchronously
  3. Off-loading findNthPrime to another thread

Throw More Nodes At It!

The industry term for “throw more resources at it” is Horizontal Scaling (as opposed to Vertical Scaling, where the name of the game is “throw better resources at it”). One of the features of Node.js that had people excited was built-in support for easy horizontal scaling via Clusters.

The general idea is to run multiple servers in parallel such that if one server is busy, another can take the incoming request. A pitfall with this approach is that it can bury the issue until the load catches up.

In our server implementation, the synchronous is slow to complete. If there is one node, it takes one request to put it out of commission. Scaling up the number of nodes will increase the capacity for those requests by the same number.

This approach is straightforward to implement but doesn’t avoid blocking the event loop; it simply adds more event loops into the mix. As a strategy, it works as long as the rate of incoming requests does not exceed the time it takes to process them.

Refactoring to Perform Work Asynchronously

Asynchronous work is usually not CPU-bound. For example, if it takes 10ms to read a file, it is likely that less than 1ms was spent waiting for the CPU, and the rest was waiting for the disk.

Calculating primes, on the other hand, is entirely CPU-bound: it’s just basic mathematical operations.

On an event loop architecture, a long-running algorithm can be converted to an asynchronous job by chunking the work onto the event loop.

Consider the following findNthPrime implementation:

const findNthPrime = num => {
  let i, primes = [2, 3], n = 5;
  const isPrime = n => {
    let i = 1, p = primes[i],
      limit = Math.ceil(Math.sqrt(n));
    while (p <= limit) {
      if (n % p === 0) {
        return false;
      }
      i += 1;
      p = primes[i];
    }
    return true;
  }
  for (i = 2; i <= num; i += 1) {
    while (!isPrime(n)) {
      n += 2;
    }
    primes.push(n);
    n += 2;
  };
  return primes[num - 1];
}
  
gantt
    title Execution Profile Of findNthPrime()
    dateFormat HH
    axisFormat %H
    todayMarker off
    section User Code
      findNthPrime() :01, 09
      isPrime()        :01, 03
      isPrime()        :03, 05
      isPrime()        :05, 07
      isPrime()        :07, 09
    section Event Loop
      Background Work    :milestone, crit, 01, sameline
      doWork()              :01, 09
      Background Work    :milestone, crit, 09, sameline

The fundamental goal of this approach is to add gaps between blocks of synchronous execution, allowing the event loop to run while your algorithm executes. Where you want those gaps to appear depends on the performance profile you are looking for. If your algorithm blocks the event loop for over a second, adding gaps anywhere is worthwhile.

In this case, isPrime() does most of the work over multiple iterations. It is already conveniently isolated in a function, which makes it a prime candidate to defer it on the event loop.

  
gantt
    title Target Execution Profile of findNthPrimeAsync()
    dateFormat HH
    axisFormat %H
    todayMarker off
    section User Code
      findNthPrime() :01, 09
      isPrime()        :01, 03
      isPrime()        :03, 05
      isPrime()        :05, 07
      isPrime()        :07, 09
    section Event Loop
      Background Work    :milestone, crit, 01, sameline
      doWork()              :01, 03
      Background Work    :milestone, crit, 03, sameline
      doWork()              :03, 05
      Background Work    :milestone, crit, 05, sameline
      doWork()              :05, 07
      Background Work    :milestone, crit, 07, sameline
      doWork()              :07, 09
      Background Work    :milestone, crit, 09, sameline

Promisify

The first step is to isolate the portion of the code to move onto the event loop into a Promise:

  const isPrime = n => new Promise(
    resolve => {
      let i = 1, p = primes[i],
        limit = Math.ceil(Math.sqrt(n));
      while (p <= limit) {
        if (n % p === 0) {
          return resolve(false);
        }
        i += 1;
        p = primes[i];
      }
      return resolve(true);
    }
  )
  // ...
  while (!await isPrime(n)) {
  //...

Turning sync code into a Promise does not make code asynchronous. For code to be asynchronous, it must be called from the event loop. setImmediate accepts a callback to do precisely that:

  const isPrime = n => new Promise(
    resolve => setImmediate(() => {
      let i = 1, p = primes[i],
        limit = Math.ceil(Math.sqrt(n));
      while (p <= limit) {
        if (n % p === 0) {
          return resolve(false);
        }
        i += 1;
        p = primes[i];
      }
      return resolve(true);
    }
  ))

Complete Implementation

const asyncInterval = setInterval(() => {
  console.log("Event loop executed");
  exCount++;
}, 1);
const findNthPrimeAsync = async num => {
  let i, primes = [2, 3], n = 5;
  const isPrime = n => new Promise(
    resolve => setImmediate(() => {
      let i = 1, p = primes[i],
        limit = Math.ceil(Math.sqrt(n));
      while (p <= limit) {
        if (n % p === 0) {
          return resolve(false);
        }
        i += 1;
        p = primes[i];
      }
      return resolve(true);
    }
  ));
  for (i = 2; i <= num; i += 1) {
    while (!await isPrime(n)) {
      n += 2;
    }
    primes.push(n);
    n += 2;
  };
  return primes[num - 1];
}

To prove that the code is now indeed on event loop, we can try scheduling on tasks on the event loop to see if they get executed:

console.log("Calculating Sync Prime...")
let syncCount = 0;
const syncInterval = setInterval(() => {
  console.log("Event loop executed");
  exCount++;
}, 1);

const sync = findNthPrime(nth);
console.log("Sync Prime is", sync)
clearInterval(syncInterval);
console.log("Intervals on event loop:", syncCount)

console.log("Calculating Async Prime...")
let asyncCount = 0;
const asyncInterval = setInterval(() => {
  console.log("Event loop executed");
  asyncCount++;
}, 1);

findNthPrimeAsync(nth)
  .then(n => console.log("Async Prime is", n))
  .then(() => clearInterval(asyncInterval))
  .then(() => console.log("Intervals on event loop:", asyncCount));

It outputs:

Calculating Sync Prime...
Sync Prime is 541
Intervals on event loop: 0
Calculating Async Prime...
Event loop executed
Event loop executed
Event loop executed
Event loop executed
Event loop executed
Event loop executed
Async Prime is 541
Intervals on event loop: 6

For a visual, the execution profile looks like this:

  
gantt
    title Execution Profile Of findNthPrime vs findNthPrimeAsync()
    dateFormat HH
    axisFormat %H
    todayMarker off
    section User Code
      findNthPrime() :01, 09
      isPrime()        :01, 03
      isPrime()        :03, 05
      isPrime()        :05, 07
      isPrime()        :07, 09
      findNthPrimeAsync() :09, 17
      isPrime()        :09, 11
      isPrime()        :11, 13
      isPrime()        :13, 15
      isPrime()        :15, 17
    section Timers
      interval :milestone, active, 11, sameline
      interval :milestone, active, 13, sameline
      interval :milestone, active, 15, sameline
    section Event Loop
      Background Work    :milestone, crit, 01, sameline
      doWork()              :01, 09
      Background Work    :milestone, crit, 09, sameline
      doWork()              :09, 11
      Background Work    :milestone, crit, 11, sameline
      doWork()              :11, 13
      Background Work    :milestone, crit, 13, sameline
      doWork()              :13, 15
      Background Work    :milestone, crit, 15, sameline
      doWork()              :15, 17
      Background Work    :milestone, crit, 17, sameline


Replit Interactive Demo

Off-Loading to Another Thread

The last approach to processing a synchronous job without blocking the main thread is offloading it to another thread entirely. Worker pools optimize this strategy further.

The premise is to have a main thread dispatch a worker:

const nth = 100; // play with this value

const findNthPrimeWorker = num => new Promise(resolve => {
  const worker = new Worker(require.resolve('./worker.js'), {
    workerData: num
  });

  worker.on("message", d => resolve(d));
})

findNthPrimeWorker(nth)

with the worker performing the computation and sending the result:

// worker.js

const findNthPrime = num => {
  // ...
}

parentPort.postMessage(findNthPrime(workerData));
  
gantt
    title Execution Profile of findNthPrimeWorker()
    dateFormat HH
    axisFormat %H
    todayMarker off
    section User Code
      findNthPrimeWorker() :01, 09
    section Event Loop
      Background Work    :milestone, crit, 01, sameline
      doWork()              :01, 02
      Background Work    :milestone, crit, 02, sameline
      doWork()              :02, 03
      Background Work    :milestone, crit, 03, sameline
      doWork()              :03, 04
      Background Work    :milestone, crit, 04, sameline
      doWork()              :04, 05
      Background Work    :milestone, crit, 05, sameline
      doWork()              :05, 06
      Background Work    :milestone, crit, 06, sameline
      doWork()              :06, 07
      Background Work    :milestone, crit, 07, sameline
      doWork()              :07, 08
      Background Work    :milestone, crit, 08, sameline
      doWork()              :08, 09
      Background Work    :milestone, crit, 09, sameline
    section Worker Thread
      findNthPrime() :01, 09
      isPrime()        :01, 03
      isPrime()        :03, 05
      isPrime()        :05, 07
      isPrime()        :07, 09

Replit Interactive Demo

Workers Limitations

Workers are ideal for moving long-running, CPU-bound tasks off the main thread but are not a silver bullet. Their primary limitation is the data that can be sent to them. The restrictions are documented in port.postMessage()

Workers != Magic!

An important callout with Workers is that by dedicating them to CPU-bound tasks, they are limited to the number of available threads. If your server has eight threads, running over eight workers will not make them run faster.

The benefit of Workers isn’t infinite parallelism but the guarantee that your main thread will always be free to perform quick work by off-loading non-so-quick work.