@ms-cloudpack/worker-pool

@ms-cloudpack/worker-pool

Provides utilities for simplifying worker execution.

Usage

There are two source files when using workers - the worker entry, which hosts the code to be executed within the worker, and the host source, which instantiates the pool and executes the work.

⚠️ Warning

For worker reuse to work properly, the method(s) being run must not have any async/scheduled code which continues running after the result is returned. Otherwise, if the scheduled code causes an error after the method returns, the error will either be swallowed (if no method was running in the worker at the time) or associated with the wrong method call (if another method call has been started).

If running arbitrary external code in a worker, it's recommended to overwrite all methods for async scheduling (setTimeout etc) with no-op versions.

Also be mindful of any cleanup of global state which might need to happen in between runs. This can be done with afterEach (see below).

Setting up a worker entry

First, let's set up the worker entry that will execute work inside the worker. Use initializeWorker to set up your api surface:

workerEntry.ts:

import { initializeWorker } from '@ms-cloudpack/worker-pool';
import { methodA, methodB } from './methods';

// Load any environmental side effects here.

// Then initialize the worker with the appropriate listeners and api dictionary.
initializeWorker({
  beforeEach: () => {...}, // optional
  afterEach: () => {...}, // optional
  methods: {
    methodA,
    methodB,
  },
});

Setting up the pool

In the host where we want to execute things to asynchronously run in the pool of workers:

import { WorkerPool } from '@ms-cloudpack/worker-pool';

const pool = new WorkerPool({ entry: './workerEntry.js' });

Executing work

Work is executed from the pool created above:

const result = await pool.execute('methodA', [arg1, arg2, etc]);