Security News
pnpm 10.0.0 Blocks Lifecycle Scripts by Default
pnpm 10 blocks lifecycle scripts by default to improve security, addressing supply chain attack risks but sparking debate over compatibility and workflow changes.
A minimal and tiny Node.js Worker Thread Pool implementation, a fork of piscina, but with fewer features
The tinypool npm package is a minimalistic thread pool implementation for Node.js. It allows you to offload tasks to worker threads, which can improve the performance of CPU-intensive operations in a Node.js application by taking advantage of multi-threading.
Creating a thread pool
This feature allows you to create a thread pool with a specified number of worker threads. The 'task' option points to a JavaScript file that will be executed in the worker threads.
const { StaticPool } = require('tinypool');
const pool = new StaticPool({
size: 4,
task: './worker.js'
});
Running tasks in the pool
Once a thread pool is created, you can run tasks by passing data to the 'run' method. The method returns a promise that resolves with the result of the task executed in the worker thread.
pool.run({ some: 'data' }).then(result => {
console.log(result);
});
Destroying the pool
When you are done with the thread pool, you can call the 'destroy' method to terminate all the worker threads and free up resources.
pool.destroy();
Workerpool is an npm package that offers a pool of workers to offload tasks. It provides a similar functionality to tinypool but with additional features such as automatic load balancing and the ability to directly pass functions and parameters to the pool without needing separate worker files.
Piscina is a modern thread pool implementation for Node.js. It is similar to tinypool but offers a more extensive API, including support for transferable objects, task cancellation, and more advanced configuration options.
Threads is a package for working with Web Workers and Node.js worker threads. It provides an abstraction over the native worker threads API, making it easier to work with. It offers a different approach compared to tinypool by focusing on the abstraction layer rather than just providing a thread pool.
Piscina: A fast, efficient Node.js Worker Thread Pool implementation
Tinypool is a fork of piscina. What we try to achieve in this library, is to eliminate some dependencies and features that our target users don't need (currently, our main user will be Vitest). Tinypool's install size (38KB) can then be smaller than Piscina's install size (6MB). If you need features like utilization or NAPI, Piscina is a better choice for you. We think that Piscina is an amazing library, and we may try to upstream some of the dependencies optimization in this fork.
✅ Smaller install size, 38KB
✅ Minimal
✅ No dependencies
✅ Physical cores instead of Logical cores with physical-cpu-count
✅ Supports worker_threads
and child_process
❌ No utilization
❌ No NAPI
Written in TypeScript, and ESM support only. For Node.js 18.x and higher.
In case you need more tiny libraries like tinypool or tinyspy, please consider submitting an RFC
node:worker_threads
// main.mjs
import Tinypool from 'tinypool'
const pool = new Tinypool({
filename: new URL('./worker.mjs', import.meta.url).href,
})
const result = await pool.run({ a: 4, b: 6 })
console.log(result) // Prints 10
// Make sure to destroy pool once it's not needed anymore
// This terminates all pool's idle workers
await pool.destroy()
// worker.mjs
export default ({ a, b }) => {
return a + b
}
// main.mjs
import Tinypool from 'tinypool'
import { MessageChannel } from 'node:worker_threads'
const pool = new Tinypool({
filename: new URL('./worker.mjs', import.meta.url).href,
})
const { port1, port2 } = new MessageChannel()
const promise = pool.run({ port: port1 }, { transferList: [port1] })
port2.on('message', (message) => console.log('Main thread received:', message))
setTimeout(() => port2.postMessage('Hello from main thread!'), 1000)
await promise
port1.close()
port2.close()
// worker.mjs
export default ({ port }) => {
return new Promise((resolve) => {
port.on('message', (message) => {
console.log('Worker received:', message)
port.postMessage('Hello from worker thread!')
resolve()
})
})
}
node:child_process
// main.mjs
import Tinypool from 'tinypool'
const pool = new Tinypool({
runtime: 'child_process',
filename: new URL('./worker.mjs', import.meta.url).href,
})
const result = await pool.run({ a: 4, b: 6 })
console.log(result) // Prints 10
// worker.mjs
export default ({ a, b }) => {
return a + b
}
// main.mjs
import Tinypool from 'tinypool'
const pool = new Tinypool({
runtime: 'child_process',
filename: new URL('./worker.mjs', import.meta.url).href,
})
const messages = []
const listeners = []
const channel = {
onMessage: (listener) => listeners.push(listener),
postMessage: (message) => messages.push(message),
}
const promise = pool.run({}, { channel })
// Send message to worker
setTimeout(
() => listeners.forEach((listener) => listener('Hello from main process')),
1000
)
// Wait for task to finish
await promise
console.log(messages)
// [{ received: 'Hello from main process', response: 'Hello from worker' }]
// worker.mjs
export default async function run() {
return new Promise((resolve) => {
process.on('message', (message) => {
// Ignore Tinypool's internal messages
if (message?.__tinypool_worker_message__) return
process.send({ received: message, response: 'Hello from worker' })
resolve()
})
})
}
We have a similar API to Piscina, so for more information, you can read Piscina's detailed documentation and apply the same techniques here.
isolateWorkers
: Disabled by default. Always starts with a fresh worker when running tasks to isolate the environment.terminateTimeout
: Disabled by default. If terminating a worker takes terminateTimeout
amount of milliseconds to execute, an error is raised.maxMemoryLimitBeforeRecycle
: Disabled by default. When defined, the worker's heap memory usage is compared against this value after task has been finished. If the current memory usage exceeds this limit, worker is terminated and a new one is started to take its place. This option is useful when your tasks leak memory and you don't want to enable isolateWorkers
option.runtime
: Used to pick worker runtime. Default value is worker_threads
.
worker_threads
: Runs workers in node:worker_threads
. For main thread <-> worker thread
communication you can use MessagePort
in the pool.run()
method's transferList
option. See example.child_process
: Runs workers in node:child_process
. For main thread <-> worker process
communication you can use TinypoolChannel
in the pool.run()
method's channel
option. For filtering out the Tinypool's internal messages see TinypoolWorkerMessage
. See example.cancelPendingTasks()
: Gracefully cancels all pending tasks without stopping or interfering with on-going tasks. This method is useful when your tasks may have side effects and should not be terminated forcefully during task execution. If your tasks don't have any side effects you may want to use { signal }
option for forcefully terminating all tasks, including the on-going ones, instead.recycleWorkers(options)
: Waits for all current tasks to finish and re-creates all workers. Can be used to force isolation imperatively even when isolateWorkers
is disabled. Accepts { runtime }
option as argument.workerId
: Each worker now has an id ( <= maxThreads
) that can be imported from tinypool
in the worker itself (or process.__tinypool_state__.workerId
). Mohammad Bagher |
---|
Your sponsorship can make a huge difference in continuing our work in open source!
The Vitest team for giving me the chance of creating and maintaing this project for vitest.
Piscina, because Tinypool is not more than a friendly fork of piscina.
FAQs
A minimal and tiny Node.js Worker Thread Pool implementation, a fork of piscina, but with fewer features
The npm package tinypool receives a total of 3,580,926 weekly downloads. As such, tinypool popularity was classified as popular.
We found that tinypool demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 3 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
pnpm 10 blocks lifecycle scripts by default to improve security, addressing supply chain attack risks but sparking debate over compatibility and workflow changes.
Product
Socket now supports uv.lock files to ensure consistent, secure dependency resolution for Python projects and enhance supply chain security.
Research
Security News
Socket researchers have discovered multiple malicious npm packages targeting Solana private keys, abusing Gmail to exfiltrate the data and drain Solana wallets.