Security News
tea.xyz Spam Plagues npm and RubyGems Package Registries
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
async-await-queue
Advanced tools
Changelog
[2.1.4] 2023-03-21
npm
dependencyprepare
scriptReadme
Promise-based priority queues for throttling, rate- and concurrency limiting of Node.js or browser tasks
Zero-dependency, total size: 2.93 kB
uncompressed and 1.16 kB
gzip-compressed
There is a medium story about using this package to parallelize download loops : Parallelizing download loops in JS with async-await-queue
This is an interesting solution to the priority queues problem.
There are other Promise-based queues out there but they are not async/await compatible and do not support priorities.
It guarantees order and never wakes up contexts that won't run.
I use it with tens of thousands of jobs on the queue. O(log(n)) on the number of jobs, O(log(n)) on the number of different priorities. Just make sure to always call Queue.end()
. Or, since 1.2, there is a safer, but less versatile method, Queue.run()
.
Typical uses:
The queues keep references to the Promise resolve()
function and resolve it from outside of the Promise constructor.
This is a very unusual use of Promises to implement locks that I find interesting (this is what the medium story is about).
npm install --save async-await-queue
Require as CJS
const { Queue } = require('async-await-queue');
Import as ES6 Module
import { Queue } from 'async-await-queue';
(or read the jsdoc)
IMPORTANT Keep in mind that when running asynchronous code without explicitly await
ing it, you should always handle the eventual Promise rejections by a .catch()
statement.
const { Queue } = require('async-await-queue');
/**
* No more than 2 concurrent tasks with
* at least 100ms between two tasks
* (measured from task start to task start)
*/
const myq = new Queue(2, 100);
const myPriority = -1;
/**
* This function will launch all tasks and will
* wait for them to be scheduled, returning
* only when all tasks have finished
*/
async function downloadTheInternet() {
for (let site of Internet) {
/**
* The third call will wait for the previous two to complete
* plus the time needed to make this at least 100ms
* after the second call
* The first argument needs to be unique for every
* task on the queue
*/
const me = Symbol();
/* We wait in the line here */
await myq.wait(me, myPriority);
/**
* Do your expensive async task here
* Queue will schedule it at
* no more than 2 requests running in parallel
* launched at least 100ms apart
*/
download(site)
/* Signal that we are finished */
/* Do not forget to handle the exceptions! */
.catch((e) => console.error(e))
.finally(() => myq.end(me));
}
return await myq.flush();
}
/**
* This is the new style API introduced in 1.2
* It is equivalent to the previous example
*/
async function downloadTheInternet() {
const q = [];
for (let site of Internet) {
/** The third call will wait for the previous two to complete
* plus the time needed to make this at least 100ms
* after the second call
*/
q.push(myq.run(() => download(site).catch((e) => console.error(e))));
}
return Promise.all(q);
}
/**
* This function will execute a single task at a time
* waiting for its place in the queue
*/
async function downloadTheInternet() {
let p;
/**
* The third call will wait for the previous two to complete
* plus the time needed to make this at least 100ms
* after the second call
* The first argument needs to be unique for every
* task on the queue
*/
const me = Symbol();
/* We wait in the line here */
await myq.wait(me, myPriority);
/**
* Do your expensive async task here
* Queue will schedule it at
* no more than 2 requests running in parallel
* launched at least 100ms apart
*/
try {
await download(site);
} catch (e) {
console.error(e);
} finally {
/* Signal that we are finished */
/* Do not forget to handle the exceptions! */
myq.end(me);
}
}
/**
* This function will schedule all the tasks and
* then will return immediately a single Promise
* that can be awaited upon
*/
async function downloadTheInternet() {
const q = [];
for (let site of Internet) {
/**
* The third call will wait for the previous two to complete
* plus the time needed to make this at least 100ms
* after the second call
* The first argument needs to be unique for every
* task on the queue
*/
const me = Symbol();
q.push(
myq
.wait(me, myPriority)
.then(() => download(site))
.catch((e) => console.error(e))
.finally(() => myq.end(me))
);
}
return Promise.all(q);
}
Promise
s in Node.jsWhen using this package, something that you should be aware of is that Node.js has a very particular behavior when dealing with unresolvable Promise
s:
https://github.com/nodejs/node/issues/43162
When await
ing an unresolvable Promise
, Node.js will simply exit - instead of blocking indefinitely - which would probably be what most people expect.
If you are using this package in Node.js and it seems to simply unexpectedly exit without reaching the program's normal end and without reporting any errors, you most probably have an unresolvable Promise
.
FAQs
async/await simple priority queues
The npm package async-await-queue receives a total of 4,313 weekly downloads. As such, async-await-queue popularity was classified as popular.
We found that async-await-queue demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
Security News
As cyber threats become more autonomous, AI-powered defenses are crucial for businesses to stay ahead of attackers who can exploit software vulnerabilities at scale.
Security News
UnitedHealth Group disclosed that the ransomware attack on Change Healthcare compromised protected health information for millions in the U.S., with estimated costs to the company expected to reach $1 billion.