parallel-task-runner

parallel-task-runner 🚀

This package is a simple and flexible parallel task queue for promise-based jobs, written in TypeScript. It allows you to control the concurrency of your asynchronous tasks, add delays between task executions, and automatically retry failed tasks. It's lightweight and has zero dependencies 🙌.

Key Features

• Concurrency Control: Limit the number of tasks that run in parallel.
• Task Prioritization: Add tasks to the front or back of the queue.
• Delayed Execution: Introduce a delay between the start of each task.
• Automatic Retries: Automatically retry tasks that fail.
• Promise-based: Works with promise-returning functions.
• Event-friendly: Emits events for idle, empty, and error states.

Table of Contents

• Installation
• Usage
  • Error Handling
• API
  • Constructor
    • new TaskQueue(config?: TaskQueueConfig)
  • Methods
    • queue.push(...tasks: Runnable[]): void
    • queue.unshift(...tasks: Runnable[]): void
    • queue.clear(): void
    • queue.waitForIdle(): Promise<IdleEventDetail>
    • queue.waitForEmpty(): Promise<void>
  • Events
    • idle: (detail: IdleEventDetail)
    • empty: (void)
    • error: (error: unknown, job: JobInfo)
• License

Installation

npm install parallel-task-runner

Usage

First, import and create a new TaskQueue instance with your desired configuration.

import TaskQueue from 'parallel-task-runner';

const queue = new TaskQueue({
	concurrency: 2, // Run a maximum of 2 tasks at once
	delay: 1000, // Wait 1 second between each task
});

Then, add tasks to the queue using push() or unshift(). A task can be a function, an async function, or function that returns a promise.

queue.push(
	async () => await setTimeout(1000), // Task 1
	() => console.log('Task 2'), // Task 2
	() => new Promise((resolve) => setTimeout(resolve, 500)), // Task 3
);

You can listen for events to know when the queue is idle, empty, or when a task has failed.

queue.on('idle', ({ pendingTasks }) => {
	console.log(`The queue is now idle. Pending tasks: ${pendingTasks}`);
});
queue.on('empty', () => {
	console.log('All tasks have been run.');
});
queue.on('error', (error, job) => {
	console.warn(`Task failed:`, job.task);
	console.error(`Error:`, error);
});

The idle and empty events also have promise-based versions.

const { pendingTasks } = await queue.waitForIdle();
console.log(`The queue is now idle. Pending tasks: ${pendingTasks}`);

await queue.waitForEmpty();
console.log('All tasks have been run.');

Error Handling

The queue can automatically retry failed tasks.

const queue = new TaskQueue({
	retryCount: 2, // Retry failed tasks 2 times
	retryDelay: 500, // Wait 500ms before retrying
});

queue.push(() => {
	return new Promise((resolve, reject) => {
		if (Math.random() > 0.5) {
			resolve();
		} else {
			reject(new Error('Task failed!'));
		}
	});
});

API

Constructor

new TaskQueue(config?: TaskQueueConfig)

Creates a new task queue.

Configuration:

• concurrency (optional, default: Infinity)
  • The maximum number of tasks to run at once.
• delay (optional, default: 0)
  • The minimum delay, in ms, between each task start.
• retryCount (optional, default: 3)
  • The maximum number of times to retry a failed task before reporting an error. Set to 0 to disable.
• retryDelay (optional, default: 0)
  • The time, in ms, to delay a retry attempt. The total delay will be at least retryDelay + delay.

Methods

queue.push(...tasks: Runnable[]): void

Schedules new tasks by appending them to the end of the queue.

• tasks: New tasks to run. A task is a function that returns a promise or void.

queue.unshift(...tasks: Runnable[]): void

Schedules new tasks by inserting them at the front of the queue.

• tasks: New tasks to run. A task is a function that returns a promise or void.

queue.clear(): void

Clears the queue, removing all pending (unstarted) tasks. Does not stop any tasks that have already started.

queue.waitForIdle(): Promise<IdleEventDetail>

A promise-based version of the idle event. Returns a promise that resolves once there are no currently running tasks. However, there may still be pending tasks waiting to run. For more details, see the idle event.

queue.waitForEmpty(): Promise<void>

A promise-based version of the idle event. Returns a promise that resolves once there are no currently running tasks and the queue is empty.

Events

• idle: Emitted when the queue is idle (no active tasks). However, there may still be pending tasks waiting to run.
• empty: Emitted when the queue is idle and there are no pending tasks left.
• error: Emitted when a task fails after all retry attempts.

idle: (detail: IdleEventDetail)

Emitted when there are no currently running tasks. However, there may still be pending tasks waiting to run. For example, with delay set to 500 ms, if two tasks are added and the first task only takes 100ms to run, then the queue will be idle for 400ms. The event provides an object with a pendingTasks property, which indicates the number of scheduled tasks in the queue.

empty: (void)

Emitted when there are no currently running tasks and the queue is empty.

error: (error: unknown, job: JobInfo)

Emitted when a task fails after all retry attempts have been exhausted. The event provides the error thrown by the task and a job object containing metadata about the failed task.

The job object has the following properties:

• task: The original task function that failed.
• retriesLeft: The number of retries remaining for this task (will always be 0).
• delayUntil: The timestamp when the last retry was scheduled.

License

This project is licensed under the MIT License.