yakugen

About

yakugen (yaku-gen) is a dynamic promise executor for Node.js. yakugen dynamically adjusts Promise concurrency to maximize performance whilst meeting target CPU & Event Loop Utilization metrics.

API Reference: https://ragrag.github.io/yakugen

Installing

npm install yakugen

Examples

Usage

async function asyncTask(item){
    await doSomething(item);
}

const items = new Array(1000).fill(Math.random());
const tasks = items.map(it => async () => {
    return asyncTask(it);
});

const resAll = await Yakugen.all(tasks);
const resAllSettled = await Yakugen.allSettled(tasks);

Controlling Concurrency Limits

async function asyncTask(item){
    await doSomething(item);
}

const items = new Array(1000).fill(Math.random());
const tasks = items.map(it => async () => {
    return asyncTask(it);
});

const res = await Yakugen.all(tasks, { minConcurrency: 5, maxConcurrency: 50 });

Yakugen defaults:

• minConcurrency: 1
• maxConcurrency: 500

Controlling Metrics Targets

async function asyncTask(item){
    await doSomething(item);
}

const items = new Array(1000).fill(Math.random());
const tasks = items.map(it => async () => {
    return asyncTask(it);
});

const res = await Yakugen.all(tasks, { targets: { cpuUtilization: 60, eventLoopUtilization: 65, eventLoopDelayMs: 100 } });

Yakugen defaults:

• cpuUtilization: 75
• eventLoopUtilization: 75
• eventLoopDelayMs: 150

Custom Metrics Targets

async function asyncTask(item){
    await doSomething(item);
}

const items = new Array(1000).fill(Math.random());
const tasks = items.map(it => async () => {
    return asyncTask(it);
});

const res = await Yakugen.all(tasks, {
    targets: {
        custom: {
            connectionPool: {
                current: () => db.currentConnections(),
                target: db.poolSize() / 2,
            },
            // ... add more custom metrics
        },
    },
});

In addition to cpuUtilization, eventLoopUtilization, eventLoopDelayMs Yakugen will also adjust concurrency based on provided custom metrics.

Track Progress

async function asyncTask(item){
    await doSomething(item);
}

const items = new Array(1000).fill(Math.random());
const tasks = items.map(it => async () => {
    return asyncTask(it);
});

const res = await Yakugen.all(tasks, {
    onProgress: (processed, metrics, currentConcurrency) => {
        console.log(processed, metrics, currentConcurrency);
    },
});