jest-worker
Advanced tools
Module for executing heavy tasks under forked processes in parallel, by providing a `Promise` based interface, minimum overhead, and bound workers.
The jest-worker npm package is a module designed to make it easy to move heavy processing tasks to child processes, thus enabling better use of multi-core systems and improving the performance of Node.js applications. It is primarily used to parallelize work across processes and manage communication between the main thread and worker threads.
Creating a Worker Pool
This feature allows you to create a pool of workers that can execute functions from a specified module file. The 'heavyTask.js' module would export the functions that you want to run on separate threads.
const { Worker } = require('jest-worker');
const worker = new Worker(require.resolve('./heavyTask.js'));
const result = await worker.myHeavyTask(args);Sending and Receiving Data
With jest-worker, you can send data to worker threads for processing and receive the processed data back in the main thread. The 'dataProcessor.js' module would contain the 'processData' function that handles the data processing logic.
const { Worker } = require('jest-worker');
const worker = new Worker(require.resolve('./dataProcessor.js'));
const processedData = await worker.processData(rawData);Ending Worker Processes
This feature allows you to properly shut down the worker processes once you're done with them, freeing up system resources.
const { Worker } = require('jest-worker');
const worker = new Worker(require.resolve('./task.js'));
// ... use the worker
await worker.end();The 'workerpool' package is similar to 'jest-worker' in that it also allows you to offload tasks to a pool of workers. It provides a different API and additional features like timeouts and the ability to cancel tasks.
The 'threads' package is another alternative that offers a high-level abstraction for working with Web Workers or Node's Worker Threads. It has a different API design and supports transferable objects for efficient data transfer.
The 'tiny-worker' package is a minimalistic implementation of the Web Workers API for Node.js. It is lightweight and simple to use but does not offer the same level of functionality or control over worker management as 'jest-worker'.
Module for executing heavy tasks under forked processes in parallel, by
providing a Promise based interface, minimum overhead, and bound workers.
The module works by providing an absolute path of the module to be loaded in all
forked processes. Files relative to a node module are also accepted. All methods
are exposed on the parent process as promises, so they can be await'ed. Child
(worker) methods can either be synchronous or asynchronous.
The module also implements support for bound workers. Binding a worker means
that, based on certain parameters, the same task will always be executed by the
same worker. The way bound workers work is by using the returned string of the
computeWorkerKey method. If the string was used before for a task, the call
will be queued to the related worker that processed the task earlier; if not, it
will be executed by the first available worker, then sticked to the worker that
executed it; so the next time it will be processed by the same worker. If you
have no preference on the worker executing the task, but you have defined a
computeWorkerKey method because you want some of the tasks to be sticked,
you can return null from it.
The list of exposed methods can be explicitly provided via the exposedMethods
option. If it is not provided, it will be obtained by requiring the child module
into the main process, and analyzed via reflection. Check the "minimal example"
section for a valid one.
$ yarn add jest-worker
This example covers the minimal usage:
parent.jsimport Worker from 'jest-worker';
async function main() {
const worker = new Worker(require.resolve('./worker'));
const result = await worker.hello('Alice'); // "Hello, Alice"
}
main();
worker.jsexport function hello(param) {
return 'Hello, ' + param;
}
The only exposed method is a constructor (Worker) that is initialized by
passing the worker path, plus an options object.
workerPath: string (required)Node module name or absolute path of the file to be loaded in the child
processes. Use require.resolve to transform a relative path into an absolute
one.
options: Object (optional)exposedMethods: $ReadOnlyArray<string> (optional)List of method names that can be called on the child processes from the parent
process. You cannot expose any method named like a public Worker method, or
starting with _. If you use method auto-discovery, then these methods will not
be exposed, even if they exist.
numWorkers: number (optional)Amount of workers to spawn. Defaults to the number of CPUs minus 1.
maxRetries: number (optional)Maximum amount of times that a dead child can be re-spawned, per call. Defaults
to 3, pass Infinity to allow endless retries.
forkOptions: Object (optional)Allow customizing all options passed to childProcess.fork. By default, some
values are set (cwd, env and execArgv), but you can override them and
customize the rest. For a list of valid values, check
the Node documentation.
computeWorkerKey: (method: string, ...args: Array<any>) => ?string (optional)Every time a method exposed via the API is called, computeWorkerKey is also
called in order to bound the call to a worker. This is useful for workers that
are able to cache the result or part of it. You bound calls to a worker by
making computeWorkerKey return the same identifier for all different calls. If
you do not want to bind the call to any worker, return null.
The callback you provide is called with the method name, plus all the rest of the arguments of the call. Thus, you have full control to decide what to return. Check a practical example on bound workers under the "bound worker usage" section.
By default, no process is bound to any worker.
The returned Worker instance has all the exposed methods, plus some additional
ones to interact with the workers itself:
getStdout(): ReadableReturns a ReadableStream where the standard output of all workers is piped.
Note that the silent option of the child workers must be set to true to make
it work. This is the default set by jest-worker, but keep it in mind when
overriding options through forkOptions.
getStderr(): ReadableReturns a ReadableStream where the standard error of all workers is piped.
Note that the silent option of the child workers must be set to true to make
it work. This is the default set by jest-worker, but keep it in mind when
overriding options through forkOptions.
end()Finishes the workers by killing all workers. No further calls can be done to the
Worker instance.
Note: Each worker has a unique id (index that starts with 1) which is
available on process.env.JEST_WORKER_ID
This example covers the standard usage:
parent.jsimport Worker from 'jest-worker';
async function main() {
const myWorker = new Worker(require.resolve('./worker'), {
exposedMethods: ['foo', 'bar', 'getWorkerId'],
numWorkers: 4,
});
console.log(await myWorker.foo('Alice')); // "Hello from foo: Alice"
console.log(await myWorker.bar('Bob')); // "Hello from bar: Bob"
console.log(await myWorker.getWorkerId()); // "3" -> this message has sent from the 3rd worker
myWorker.end();
}
main();
worker.jsexport function foo(param) {
return 'Hello from foo: ' + param;
}
export function bar(param) {
return 'Hello from bar: ' + param;
}
export function getWorkerId() {
return process.env.JEST_WORKER_ID;
}
This example covers the usage with a computeWorkerKey method:
parent.jsimport Worker from 'jest-worker';
async function main() {
const myWorker = new Worker(require.resolve('./worker'), {
computeWorkerKey: (method, filename) => filename,
});
// Transform the given file, within the first available worker.
console.log(await myWorker.transform('/tmp/foo.js'));
// Wait a bit.
await sleep(10000);
// Transform the same file again. Will immediately return because the
// transformed file is cached in the worker, and `computeWorkerKey` ensures
// the same worker that processed the file the first time will process it now.
console.log(await myWorker.transform('/tmp/foo.js'));
myWorker.end();
}
main();
worker.jsimport babel from 'babel-core';
const cache = Object.create(null);
export function transform(filename) {
if (cache[filename]) {
return cache[filename];
}
// jest-worker can handle both immediate results and thenables. If a
// thenable is returned, it will be await'ed until it resolves.
return new Promise((resolve, reject) => {
babel.transformFile(filename, (err, result) => {
if (err) {
reject(err);
} else {
resolve((cache[filename] = result));
}
});
});
}
23.0.0
[expect] Expose getObjectSubset, iterableEquality, and subsetEquality (#6210)[jest-snapshot] Add snapshot property matchers (#6210)[jest-config] Support jest-preset.js files within Node modules (#6185)[jest-cli] Add --detectOpenHandles flag which enables Jest to potentially track down handles keeping it open after tests are complete. (#6130)[jest-jasmine2] Add data driven testing based on jest-each (#6102)[jest-matcher-utils] Change "suggest to equal" message to be more advisory (#6103)[jest-message-util] Don't ignore messages with vendor anymore (#6117)[jest-validate] Get rid of jest-config dependency (#6067)[jest-validate] Adds option to inject deprecationEntries (#6067)[jest-snapshot] [BREAKING] Concatenate name of test, optional snapshot name and count (#6015)[jest-runtime] Allow for transform plugins to skip the definition process method if createTransformer method was defined. (#5999)[expect] Add stack trace for async errors (#6008)[jest-jasmine2] Add stack trace for timeouts (#6008)[jest-jasmine2] Add stack trace for thrown non-Errors (#6008)[jest-runtime] Prevent modules from marking themselves as their own parent (#5235)[jest-mock] Add support for auto-mocking generator functions (#5983)[expect] Add support for async matchers (#5919)[expect] Suggest toContainEqual (#5953)[jest-config] Export Jest's default options (#5948)[jest-editor-support] Move coverage to ProjectWorkspace.collectCoverage (#5929)[jest-editor-support] Add coverage option to runner (#5836)[jest-haste-map] Support extracting dynamic imports (#5883)[expect] Improve output format for mismatchedArgs in mock/spy calls. (#5846)[jest-cli] Add support for using --coverage in combination with watch mode, --onlyChanged, --findRelatedTests and more (#5601)[jest-jasmine2] [BREAKING] Adds error throwing and descriptive errors to it/ test for invalid arguments. [jest-circus] Adds error throwing and descriptive errors to it/ test for invalid arguments (#5558)[jest-matcher-utils] Add isNot option to matcherHint function (#5512)[jest-config] Add <rootDir> to runtime files not found error report (#5693)[expect] Make toThrow matcher pass only if Error object is returned from promises (#5670)[expect] Add isError to utils (#5670)[expect] Add inverse matchers (expect.not.arrayContaining, etc., #5517)[expect] expect.extend now also extends asymmetric matchers (#5503)[jest-mock] Update spyOnProperty to support spying on the prototype chain (#5753)[jest-mock] Add tracking of return values in the mock property (#5752)[jest-mock] Add tracking of thrown errors in the mock property (#5764)[expect]Add nthCalledWith spy matcher (#5605)[jest-cli] Add isSerial property that runners can expose to specify that they can not run in parallel (#5706)[expect] Add .toBeCalledTimes and toHaveBeenNthCalledWith aliases (#5826)[jest-cli] Interactive Snapshot Mode improvements (#5864)[jest-editor-support] Add no-color option to runner (#5909)[jest-jasmine2] Pretty-print non-Error object errors (#5980)[jest-message-util] Include column in stack frames (#5889)[expect] Introduce toStrictEqual (#6032)[expect] Add return matchers (#5879)[jest-cli] Improve snapshot summaries (#6181)[expect] Include custom mock names in error messages (#6199)[jest-diff] Support returning diff from oneline strings (#6221)[expect] Improve return matchers (#6172)[jest-cli] Overhaul watch plugin hooks names (#6249)[jest-mock] [BREAKING] Include tracked call results in serialized mock (#6244)[jest-cli] Fix stdin encoding to utf8 for watch plugins. (#6253)[expect] Better detection of DOM Nodes for equality (#6246)[jest-cli] Fix misleading action description for F key when in "only failed tests" mode. (#6167)[jest-worker] Stick calls to workers before processing them (#6073)[babel-plugin-jest-hoist] Allow using console global variable (#6075)[jest-jasmine2] Always remove node core message from assert stack traces (#6055)[expect] Add stack trace when expect.assertions and expect.hasAssertions causes test failures. (#5997)[jest-runtime] Throw a more useful error when trying to require modules after the test environment is torn down (#5888)[jest-mock] [BREAKING] Replace timestamps with invocationCallOrder (#5867)[jest-jasmine2] Install sourcemap-support into normal runtime to catch runtime errors (#5945)[jest-jasmine2] Added assertion error handling inside afterAll hook (#5884)[jest-cli] Remove the notifier actions in case of failure when not in watch mode. (#5861)[jest-mock] Extend .toHaveBeenCalled return message with outcome (#5951)[jest-runner] Assign process.env.JEST_WORKER_ID="1" when in runInBand mode (#5860)[jest-cli] Add descriptive error message when trying to use globalSetup/globalTeardown file that doesn't export a function. (#5835)[expect] Do not rely on instanceof RegExp, since it will not work for RegExps created inside of a different VM (#5729)[jest-resolve] Update node module resolution algorithm to correctly handle symlinked paths (#5085)[jest-editor-support] Update Settings to use spawn in shell option (#5658)[jest-cli] Improve the error message when 2 projects resolve to the same config (#5674)[jest-runtime] remove retainLines from coverage instrumentation (#5692)[jest-cli] Fix update snapshot issue when using watchAll (#5696)[expect] Fix rejects.not matcher (#5670)[jest-runtime] Prevent Babel warnings on large files (#5702)[jest-mock] Prevent mockRejectedValue from causing unhandled rejection (#5720)[pretty-format] Handle React fragments better (#5816)[pretty-format] Handle formatting of React.forwardRef and Context components (#6093)[jest-cli] Switch collectCoverageFrom back to a string (#5914)[jest-regex-util] Fix handling regex symbols in tests path on Windows (#5941)[jest-util] Fix handling of NaN/Infinity in mock timer delay (#5966)[jest-resolve] Generalise test for package main entries equivalent to ".". (#5968)[jest-config] Ensure that custom resolvers are used when resolving the configuration (#5976)[website] Fix website docs (#5853)[expect] Fix isEqual Set and Map to compare object values and keys regardless of order (#6150)[pretty-format] [BREAKING] Remove undefined props from React elements (#6162)[jest-haste-map] Properly resolve mocked node modules without package.json defined (#6232)[jest-runner] Move sourcemap installation from jest-jasmine2 to jest-runner (#6176)[jest-cli] Use yargs's built-in version instead of rolling our own (#6215)[docs] Add explanation on how to mock methods not implemented in JSDOM[jest-jasmine2] Simplify Env.execute and TreeProcessor to setup and clean resources for the top suite the same way as for all of the children suites (#5885)[babel-jest] [BREAKING] Always return object from transformer (#5991)[*] Run Prettier on compiled output (#5858)[jest-cli] Add fileChange hook for plugins (#5708)[docs] Add docs on using jest.mock(...) (#5648)[docs] Mention Jest Puppeteer Preset (#5722)[docs] Add jest-community section to website (#5675)[docs] Add versioned docs for v22.4 (#5733)[docs] Improve Snapshot Testing Guide (#5812)[jest-runtime] [BREAKING] Remove jest.genMockFn and jest.genMockFunction (#6173)[jest-message-util] Avoid adding unnecessary indent to blank lines in stack traces (#6211)FAQs
Module for executing heavy tasks under forked processes in parallel, by providing a `Promise` based interface, minimum overhead, and bound workers.
The npm package jest-worker receives a total of 54,816,219 weekly downloads. As such, jest-worker popularity was classified as popular.
We found that jest-worker demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 5 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
A malicious npm package disguised as a WhatsApp client is exploiting authentication flows with a remote kill switch to exfiltrate data and destroy files.
Security News
PyPI now supports digital attestations, enhancing security and trust by allowing package maintainers to verify the authenticity of Python packages.
Security News
GitHub removed 27 malicious pull requests attempting to inject harmful code across multiple open source repositories, in another round of low-effort attacks.