betterthread

betterthread

betterthread allows you to easily write JavaScript that can be securely executed in parallel across threads, CPUs or VMs. This can be used for simply moving slow tasks into another thread, or to create complex high performance multi-CPU distrobuted systems. No native modules required, this is 100% pure javascript.

There are plenty of advanced options, but basic functionality is very easy to use.

const bt = require('betterthread');

const myWorker = new bt.ThreadedFunction((message, done)=>{
    const foo = message + ' World!';
    done(foo); // pass result back to main thread
});

myWorker.on('data',(data)=>{
    console.log(`I computed '${data}' in another thread!`)
    myWorker.kill();
});

myWorker.send('Hello');

Many more examples are available.

Stream example:

const bt = require('betterthread');

const myWorker = new bt.ThreadedFunctionStream(stream => {
    stream.on('data', data => {
        stream.emit('data', 'Hello' + data.name);
    });
});

myWorker.emit('Fred');
myWorker.on('data', data => {
    console.log(data);
});

Table of Contents

1. Overview
2. FAQ
3. Classes
4. Options
5. Examples
   • sha.js: Basic iterative SHA-256 example; 500,000 iterations.
   • sandbox.js: Demonstrate V8 virtual machine contexts preventing native API usage
   • startupPerformance.js: Find code excution startup and iteration time
   • stream.js: Stream-based transfrom example
   • abortingExecution.js: Cleanly kill worker process that are stuck or failed
   • verboseMode.js: Example of verbose mode for tracking thread lifecycle
   • multipleThreads.js: Example using an array multiple threads
   • estimatePiParallel.js: Compute Pi in parallel using the Monte Carlo method
6. Known Issues

FAQ

Why do I need this?

Node.js doesn't provide a way to simply execute in another thread. Built-ins such as cluster work great for sharing a HTTP server, but don't work well for general-purpse computing.

Do I need to use the experimental Worker feature in 10.5.0?

No, this library does not require any experimental features and works on the current LTS version and old versions of Node; right now Node version 6.x.x to 10.x.x are supported.

How long does it take to spin up a new thread?

Starting a thread will take somewhere around half a second. You can test this by runningRun the following command to start this demo: node ./examples/spinupPerformance.js.

What does the CPU usage of the main thread look like?

With many of the examples, the main thread's time is only about 150mSec.

stonegray-vm2:betterthread stonegray$ ps -T -g 41943
  PID TTY           TIME CMD
41943 ttys005    0:00.15 node shaExample.js
41944 ttys005    0:06.26 /usr/local/bin/node ./betterthread/worker.js
41948 ttys006    0:00.01 ps -T -g 41943

What doesn't work in a ThreadedFunction?

Anything that can run in your main thread can run in a ThreadedFunction; there are currently two exceptions:

• process.send() and process.exit() will not work as expected; they will apply to the worker not the parent. A patch for this is planned.
• If you use the cluster library, (eg. running a multithreaded HTTP server) it will not work as expected at this time. A polyfill for cluster is planned.

Can I nest threads within threads within threads?

Not right now. See above, process.send() and cluster need to be patched first.

What platforms does this support?

This should run on any platform that is supported by Node, from TV boxes running Linux to standard desktop PCs.

What other neat things can I do with this that aren't in the examples?

• You can manually set the uid and gid of a process to restrict the thread's permissions if you're root.
• You can run your code in a V8 sandbox and only expose native APIs you specify.

What about [other library]?

Project	Comparison
hamsters.io	Browser only, does not perform true multithreading on Node.js.
napa.js	Uses native modules to achieve multithreading and does not run on arm64 architectures. About twice as fast as betterthread based on Microsoft's parallel pi computation
threads.js	Runs on both Node and the browser; betterthread currently only supports Node.

_{Found another comparible library? Add it here and submit a PR!}

Classes

bt.ThreadedFunction

Create a simple event-based inline thread.

bt.SyncThreadedFunction

Create a simple event-based inline thread optimized for native modules and JavaScript functions which return();

bt.StreamFunction

Create an inline thread with a Duplex Stream.

Usage

const bt = require('betterthread');
const myWorker = new bt.ThreadedFunction(func, options);

ThreadedFunction( ... ).start(void);

Start thread execution with no arguments. State will change to

Example:

const bt = require('betterthread');
const myWorker = new bt.ThreadedFunction(func, options);
myWorker.start();

Examples

Five examples are included to demonstrate usage of betterthread.

• sha.js: Basic iterative SHA-256 example; 500,000 iterations.
• sandbox.js: Demonstrate V8 virtual machine contexts preventing native API usage
• startupPerformance.js: Find code excution startup and iteration time
• stream.js: Stream-based transfrom example
• abortingExecution.js: Cleanly kill worker process that are stuck or failed
• verboseMode.js: Example of verbose mode for tracking thread lifecycle
• multipleThreads.js: Example using an array multiple threads
• estimatePiParallel.js: Compute Pi in parallel using the Monte Carlo method

SHA-256

Run the following command to start this demo: node ./examples/sha.js

This example performs a 250,000-iteration SHA sum on a string. Running SHA256 is CPU intensive and normally this would cause your CPU usage to go up.

This demonstrates a core usage of betterthread, to move time-consuming synchronous tasks out of the event loop.

// For a fully commented version, see `./examples/sha.js`
const bt = require('betterthread');

// Create a threaded function:
const myWorker = new bt.ThreadedFunction((message, done)=>{
    const crypto = require('crypto');
    let i = 5 * 2e5;
    while (i--){
        message = crypto.createHash('sha256').update(message, 'utf8').digest('hex');
    }
    done(message.toString('utf8'));
});

// Handle the callback:
myWorker.on('data',(data)=>{
    console.log(`Worker completed, result is ${data}`)
   myWorker.kill();
});

// Run the function in a worker:
myWorker.send('Hello');

Program Output:

stonegray-vm2:examples stonegray$ node sha.js
Worker completed, result is 6464c793dd45ad1e341670308529cc82e52524df37dd60fc6524a7a0bbaa3dba

Thread Time:

stonegray-vm2:betterthread stonegray$ ps -T -g 41943
  PID TTY           TIME CMD
41943 ttys005    0:00.15 node shaExample.js
41944 ttys005    0:06.26 /usr/local/bin/node ./betterthread/worker.js
41948 ttys006    0:00.01 ps -T -g 41943

Sandbox

Run the following command to start this demo: node ./examples/sandbox.js

This example demonstrates using a custom context to isolate the thread from specific Node APIs.

The following options are set when the thread is created. The thread tries to access the process,root and http builtins, as well as require() the fs module. You can try allowing requiring modules by adding the string require to options.exposedApis to see this check fail.

const options = {
    vm: true,
    exposedApis: ['console']
};

Program Output:

stonegray-vm2:examples stonegray$ node sandbox.js
Process object is undefined
Root object is undefined
HTTP object is undefined
Could not load filesystem module: "ReferenceError: require is not defined"

Startup

Run the following command to start this demo: node ./examples/startupPerformance.js

This example just starts a new thread and times how long it takes to do a round-trip from runtime to when a callback is recieved from the thread. This can be used to make performance decisions on thread reuse/pooling.

Output:

stonegray-vm2:examples stonegray$ node spinupPerformance.js
Startup took 0.857s (857ms)
Running a command took 0.012480553s (12.480553ms)
Running a command took 0.000642424s (0.642424ms)
Running a command took 0.000574866s (0.5748660000000001ms)
Running a command took 0.000295431s (0.295431ms)

Stream

Run the following command to start this demo: node ./examples/stream

This examples runs a worker and estabilishes a stream connection to it.

const bt = require('betterthread');

const myWorker = new bt.ThreadedFunctionStream(stream => {
    stream.on('data', data => {
        stream.emit('data', 'Hello ' + data.name);
    });
});

myWorker.emit('Fred');
myWorker.on('data', data => {
    console.log(data);
});

Output:

Hello Fred

Aborting Execution

Run the following command to start this demo: node ./examples/abortingExecution.js

This example demonstrates killing an unresponsive thread. The while(1) loop synchronly blocks execution, causing the program to hang. When it doesn't respond in 500mSec, it is killed with a human-readable reason code.

// Example
const bt = require('..'); // require('betterthread');

// Create a thread that will never return;
const thread = new bt.ThreadedFunction(() => {
    while(true) process.stdout.write('.')
},{
    verbose: true
});

// Start the thread, after 500mSec, kill it. 
thread.start();
setTimeout(()=>{
    thread.kill('Timeout');
},500);

Output:

stonegray-vm2:examples stonegray$ node abortingExecution.js
[worker] 53689 connected
[worker] 53689 state change to starting
[worker] 53689 state change to waitForJob
[worker] 53689 state change to ready
...............................................................................................................................................................................................................................................
[worker] 53689 set reason code to Timeout
..................
[worker] 53689 signal SIGTERM with reason 'Timeout'
[worker] 53689 disconnected

Verbose Mode

Run the following command to start this demo: node ./examples/verboseMode.js

This example shows the logging support, useful for debugging issues related to worker lifetime.

The following options are set:

const options = {
    verbose: true,
    description: 'My Worker Process'
};

stonegray-vm2:examples stonegray$ node multipleThreads.js
[My Worker Process] 47196 connected
[My Worker Process] 47196 state change to starting
[My Worker Process] 47196 state change to waitForJob
[My Worker Process] 47196 state change to ready
Worker returned string: Hello World
Killing worker in 500mSec...
[My Worker Process] 47196 set reason code to Test Reason
[My Worker Process] 47196 state change to ready
[My Worker Process] 47196 signal SIGTERM with reason 'Test Reason'
[My Worker Process] 47196 disconnected
stonegray-vm2:examples stonegray$

Multiple Threads

Run the following command to start this demo: node ./examples/multipleThreads.js

This example spins up multiple threads from an array. This can be used for a variety of purposes.

// Multiple Thread Example
const bt = require('betterthread')

const threadNames = [
    "counting",
    "thinking",
    "logging",
    "calculating",
    "multiplying"
];

// Create threads for each in array:
threadNames.forEach(name => {
    const thread = new bt.ThreadedFunction((message, done) => {
        done(`Hello from the ${message} thread, with PID ${process.pid}!`);
    });
    thread.on('data', (data) => {
        console.log(`${data}`)
        thread.kill();
    });
    thread.send(name);
});

console.log(`Hello from the master thread, with PID ${process.pid}`);

Output:

stonegray-vm2:examples stonegray$ node multipleThreads.js
Hello from the master thread, with PID 45577
Hello from the calculating thread, with PID 45581!
Hello from the logging thread, with PID 45580!
Hello from the counting thread, with PID 45578!
Hello from the thinking thread, with PID 45579!
Hello from the multiplying thread, with PID 45582!
stonegray-vm2:examples stonegray$

Parallel Pi

Run the following command to start this demo: node ./examples/examplePiParallel.js

Note: This example is copied from Microsoft's napa.js project with a wrapper to support betterthread

This example implements an algorithm to estimate PI using Monte Carlo method. It demonstrates how to fan out sub-tasks into multiple JavaScript threads, execute them in parallel and aggregate output into a final result.

Output:

stonegray-vm2:examples stonegray$ node estimatePiParallel.js
# of points     # of batches    # of workers    latency in MS   estimated π     deviation
---------------------------------------------------------------------------------------
10000000                1               8               511             3.141882        0.0002897464
10000000                2               8               162             3.141626        0.00003374641
10000000                4               8               109             3.140659        0.0009334536
10000000                8               8               85              3.141542        0.00005025359

When all CPU cores are used, the threads are very evenly balanced by the OS, and the main thread's CPU usage is near zero.

stonegray-vm2:examples stonegray$ ps -T -g 50467
  PID TTY           TIME CMD
47816 ttys005    0:00.13 /bin/bash -l
50467 ttys005    0:00.14 node estimatePiParallel
50468 ttys005    0:07.42 /usr/local/bin/node ../betterthread/worker.js
50469 ttys005    0:07.41 /usr/local/bin/node ../betterthread/worker.js
50470 ttys005    0:07.41 /usr/local/bin/node ../betterthread/worker.js
50471 ttys005    0:07.38 /usr/local/bin/node ../betterthread/worker.js
50472 ttys005    0:07.39 /usr/local/bin/node ../betterthread/worker.js
50473 ttys005    0:07.39 /usr/local/bin/node ../betterthread/worker.js
50474 ttys005    0:07.35 /usr/local/bin/node ../betterthread/worker.js
50475 ttys005    0:07.38 /usr/local/bin/node ../betterthread/worker.js
50483 ttys005    0:00.01 ps -T -g 50467

Options

WARNING: The advanced options at the bottom of the default option list are intended for experimentaiton or debugging and should be used with extreme caution. Many have security, performance, or reliability implications that are not documented.

Defualt options:

{
    // Enable console logging
    verbose: false,

    /* To restrict what the process can do, you can run it within a V8 Virtual Machine context. By default, a relatively permissive VM is used, but this can be tweaked. This is quite slow right now because we recreate the context each run.*/
    vm: false,
    
    /* Pass options related to V8 VM isolation; ignored if this.vm === false. */
    vmOpts: {
        /* Expose native APIs in the VM; by default, only require() and console are available. Note that this allows you to require builtins such as `fs` and `https` which may be unwanted. */
        /* If you would like to require a library outside of the VM and pass a reference in, you can do so using the advanced options below. Note the security warnings in doing this */
        expose: ['require','console']

        /* Enable experimental ES module support. Not recommended for production at this time, and test coverage is not planned.*/
        experimentalESModuleSupport: false
    },

    /* Use a custom debug port to allow IDEs to debug the remote thread using the internal Node debugger */
    debugPort: undefined,

    /* Fill newly created instances of Buffer() with zeroes for security. This is not default Node behaviour.*/
    zeroMallocBuffers: true,

    /* You can request that the child processes be spawned with a different user ID or group ID. You will recieve an EPERM if this fails. */
    uid: undefined, // Requested user ID; numeric only.
    gid: undefined,	// Requested group ID; numeric only.

    // Advanced options:
    // ////////////////////////////////

    /* Create V8 process profiling isolates in the CWD. Filename will be `isolate-${address}-v8-${PID}.log`, which can be converted using Node's performance processing utility. This file logs execution state at each tick, and can grow to be very large. */
    profile: false,

    /* Debug macro to enable all debugging features. Everytime you enable this on prod a kitten dies. */
    debug: false,

    /* Require modules before loading the worker task runner. SECURITY WARNING: This can bypass any object-based security policy implemented by the V8 VM options you set above. */
    preflightRequire: [],

    /* Apply specific arguments to the process. Use with caution. SECURITY WARNING: This can bypass any object-based security policy implemented by the V8 VM options you set above. */
    processArgs: [],
}

Technology Readiness