A simple and powerful async abstraction lib for easily writing Node.js code.
A simple and powerful async abstraction layer lib to easily write Node.js code.
While inspired in some way by caolan/async, Async Kit uses a completely different approach.
Rather than having a whole bunch of specific functions, this lib provides a generic way to solve async code flow. So anything that can be done by caolan/async lib can be converted to Async Kit, but the reverse is not always true.
Using natural syntax really easy to become familiar with, you will be able to code great things effortlessly, without cumbersome callback hell, and without coding again and again the same async pattern and logic.
Please read this doc on Github, npmjs.org truncate it.
async.series( [
function( callback ) {
letsConnectToDatabase( callback ) ;
} ,
function( callback ) {
letsQueryTheDatabase( callback ) ;
} ,
function( callback ) {
doMoreQueries( callback ) ;
}
] )
.exec( function( error , results ) {
if ( error ) { console.log( 'Doh!' ) ; }
else { console.log( 'Yay! Done!' ) ; }
} ) ;
This small example prepares an async job's list and executes it.
All jobs are executed in series, one after one.
Each callback works the Node.js way, the first argument is always the error argument.
If one job fails (ie it triggers its callback with an error or any truthy value), all remaining jobs are skipped
and the exec()'s callback is instantly called with that error.
When every jobs are finished, the exec()'s callback is called, the results argument contains an array of the arguments passed by each job to its callback.
.exec() with, that are transmitted to jobsUse Node Package Manager:
npm install async-kit
This is an important concept to understand when using this lib: there are two stages to perform an async flow.
In the first stage, you define the plan.
All plan definition returns an async.Plan object.
Then you can .exec() your plan as many time as you want. All the exec method family returns an execContext object.
The first time an async.Plan is .exec(), it becomes locked forever: you cannot modify it anymore.
The example above becomes:
// Plan stage, jobs' definition
var plan = async.series( [
function( callback ) {
letsConnectToDatabase( callback ) ;
} ,
function( callback ) {
letsQueryTheDatabase( callback ) ;
} ,
function( callback ) {
doMoreQueries( callback ) ;
}
] ) ;
// Change the plan, each job should terminate within 200ms
plan.timeout( 200 ) ;
// Exec stage
plan.exec( function( error , results ) {
if ( error ) { console.log( 'Doh!' ) ; }
else { console.log( 'Yay! Done!' ) ; }
} ) ;
plan.exec( function( error , results ) {
if ( error ) { console.log( 'Doh x2!' ) ; }
else { console.log( 'Yay! Again!' ) ; }
} ) ;
// No effect! Plan cannot be modified anymore!
plan.timeout( 200 ) ;
In most case, callbacks work in the Node.js fashion, except explicitly expressed otherwise. The callback should always be called with arguments in this order:
callback( [error] , [argument1] , [argument2] , ... ) ;
That's it: the first argument, if present, is always assumed to be the error argument.
Async Kit will assume that something is wrong with a job if it get ANY truthy value as the error argument, weither it is an instanceof of Error, true, 'my error message', or any expression evaluated to true. If you are unsure what are truthy and falsy values, check this out.
Use case: this is probably the most common use case for any website, we have to perform a series of async query, each query should be sent after the previous one succeed.
async.waterfall( [
function getUserByLoginAndPassword( login , password , callback ) {
dbUserCollection.findOne( { login: login, password: password } , callback ) ;
} ,
function getUserPhoto( userDocument , callback ) {
dbPhotoCollection.findOne( { _id: userDocument.photoID } , callback ) ;
}
] )
.timeout( 200 )
.then( function( photoDocument ) {
httpResponse.writeHead( 200 , { 'Content-Type' : 'image/png' } ) ;
httpResponse.write( photoDocument.rawData ) ;
httpResponse.end() ;
} )
.catch( function( error ) {
httpResponse.writeHead( 404 , { 'Content-Type' : 'text/plain' } ) ;
httpResponse.write( '404 - Not found.' ) ;
httpResponse.end() ;
} )
.execArgs( 'john@example.com' , 'god' ) ;
Explanation:
.exec()-like function, since by default
.exec() assume than its last argument is the finally callback, so since we are in waterfall mode, every
arguments passed to execArgs() are passed only to the first jobYou can chain as many queries as you want, without burying them deeper and deeper in nested callback hell.
Use case: we want to get some contents (JSON, HTML, RSS, etc), many mirrors are available but we don't want to try them one at a time, we want to try them all at once and finish as soon as possible, when the first non-error response is received.
async.race( [ url1 , url2 , url3 , url4 ] )
.using( function( url , callback ) {
getContentFromUrl( url , callback ) ;
} )
.then( function( contents ) {
doSomethingWithContent( contents ) ;
} )
.catch( function( error ) {
console.log( "Cannot get contents from any mirror" ) ;
} )
.exec() ;
Explanation:
callback( error , contents )Use case: we have an array, we want to iterate it but there are some async code in the iterator, and we really want that each element to be processed one at a time. The native javascript myArray.forEach() would parallelize the async part even if we don't want.
async.foreach( myArray , function( element , callback ) {
doSomethingAsyncWithElement( element , callback ) ;
} )
.exec( function( error ) {
console.log( "Finished!" ) ;
} ) ;
Explanation:
.exec()'s callback is triggered, as usualYou can as well add a .parallel() before .exec(), you still have the advantage versus native forEach()
of having a general callback triggered when everything is asynchronously done.
They create async.Plan object and set up the job's list.
Note that an async.Plan do not perform anything until its .exec() method is called (see Class async.Plan for details).
The following informations describe what happend when the plan is executed.
By default, jobs are processed one at a time.
If an error occurs, no new jobs will be processed.
Jobs should trigger their callback the Node.js way: callback( error , [arg1] , [arg2] , ... ).
The finally callbacks (see below) are triggered when the first error occurs or when all jobs are done.
Note: all factories below are described relative to this point of reference. Only differences will be reported.
Array or ObjectThis is the most generic factory, with default behaviour, with no further limitation.
See Do family factories above.
Array or ObjectSet up a job's list to be processed in series.
Calling .parallel() on it has no effect, it will process jobs one at a time anyway.
Array or ObjectSet up a job's list to be processed in parallel.
The parallel limit is set to Infinity by default.
Array or ObjectSet up a job's list to be processed in parallel.
The parallel limit is set to Infinity by default.
The whole jobs processing aborts when the first job finish without error.
Jobs processing continues on error.
Note that async.race( jobsList ) is the same than async.parallel( jobsList ).fatal( false ).race().
Array or ObjectSet up a job's list to be processed in series, in waterfall mode.
Each job is called with the previous job output as arguments.
By default, the .exec() method accept arguments to pass to the first job.
By default, the error argument is not transmitted, see .transmitError() for details.
Only the last job pass its result to finallyCallback, thenCallback etc... See .lastJobOnly() for details.
Calling .parallel() on it has no effect, it will process jobs one at a time anyway.
Example:
async.waterfall( [
function( str , callback ) {
// str equals 'oh', passed by .exec()'s first argument
callback( null , str + ' my' ) ;
// null is the error argument, it is not transmitted to the next job by default
} ,
function( str , callback ) {
// str equals 'oh my', passed by the previous job
callback( null , str + ' wonderful' ) ;
// null is the error argument, it is not transmitted to the next job by default
} ,
function( str , callback ) {
// str equals 'oh my wonderful', passed by the previous job
callback( null , str + ' result' ) ;
}
] )
.exec( 'oh' , function( error , results ) {
// output 'oh my wonderful result'
console.log( results ) ;
} ) ;
Any number of arguments can be used. The previous example can become something like this:
async.waterfall( [
function( str1 , str2 , str3 , callback ) {
// str1 equals 'Hello', passed by .exec()'s first argument
// str2 equals 'world', passed by .exec()'s second argument
// str3 equals 'this', passed by .exec()'s third argument
callback( null , str1 + ' ' + str2 + ' ' + str3 + ' is' ) ;
} ,
function( str , callback ) {
// str equals 'Hello world, this is', passed by the previous job
callback( null , str + ' my' , 'wonderful' ) ;
} ,
function( str1 , str2 , callback ) {
// str1 equals 'Hello world, this is my', passed by the previous job
// str2 equals 'wonderful', passed by the previous job
callback( null , str1 + ' ' + str2 + ' result' ) ;
}
] )
.exec( 'Hello' , 'world,' , 'this' , function( error , results ) {
// output 'Hello world, this is my wonderful result'
console.log( results ) ;
} ) ;
Array or Object to iterateFunction( element , [key] , [container] , callback ) where:
mixed the current array element or object's property valueNumber or String the current key (index for array, property name for object)Array or Object, this is the original containerFunction( error , [arg1] , [arg2] , ... ) a node-style callback to trigger on completionIt performs an async foreach, iterating container, using iterator.
Depending on iterator.length (the number of arguments the user-provided function accept), the arguments passed to iterator
will be ( element , callback ), ( element , key , callback ), or ( element , key , container , callback )
where element is the current element, key is the current key (the current index if container is an Array,
or the current property's name if container is an object), container is the original container,
and callback is the completion's callback.
By default, elements are performed one at a time, in series.
If the iterator fails for one element, it will continue processing others elements anyway.
Note that async.foreach( container , iterator ) is equal to async.do( container ).iterator( iterator ).
Example:
var myArray = [ 'one' , 'two' , 'three' ] ;
async.foreach( myArray , function( element , callback ) {
// Called three time, with element's value: 'one', then 'two', then 'three'
doSomethingAsyncWithElement( element , callback ) ;
} )
.exec( function( error , results ) {
thingsToDoWhenFinished() ;
} ) ;
Array or Object to iterateFunction( element , [key] , [container] , callback ) where:
mixed the current array element or object's property valueNumber or String the current key (index for array, property name for object)Array or Object, this is the original containerFunction( error , [arg1] , [arg2] , ... ) a node-style callback to trigger on completionIt performs an async map, iterating container, using iterator. An async map takes an array and produces a new array, each value in the input array is mapped into the output array, preserving indexes. If an object is provided instead of an array, it produces a new object, preserving keys.
Depending on iterator.length (the number of arguments the user-provided function accept), the arguments passed to iterator
will be ( element , callback ), ( element , key , callback ), or ( element , key , container , callback )
where element is the current element, key is the current key (the current index if container is an Array,
or the current property's name if container is an object), container is the original container,
and callback is the completion's callback.
By default, elements are performed in parallel mode.
If the iterator fails for one element, it will continue processing others elements anyway.
The results (see example below) directly map the container, like .mapping1to1() do.
Note that async.map( container , iterator ) is equal to async.do( container ).iterator( iterator ).mapping1to1().
Example:
var myArray = [ 'my' , 'wonderful' , 'result' ] ;
async.map( myArray , function( element , callback ) {
setTimeout( function() {
callback( null , element.length ) ;
} , 0 ) ;
} )
.exec( function( error , results ) {
// we expect results to be equal to [ 2, 9, 6 ]
expect( results ).to.be.eql( [ 2, 9, 6 ] ) ;
} ) ;
Array or Object to iteratemixed the initial default reduced (aggregated) valueFunction( aggregatedValue , element , [key] , [container] , callback ) where:
mixed the current reduced valuemixed the current array element or object's property valueNumber or String the current key (index for array, property name for object)Array or Object, this is the original containerFunction( error , newAggregatedValue , [arg1] , [arg2] , ... ) a node-style callback to trigger on completion, where:
mixed is the new reduced value that will be passed to the next iterationIt performs an async reduce, iterating container, using iterator. An async reduce takes an array (or an object), and iterate it to produce a single reduced value (though actually this single value can be anything we like, even an array or object).
Depending on iterator.length (the number of arguments the user-provided function accept), the arguments passed to iterator
will be ( aggregatedValue , element , callback ), ( aggregatedValue , element , key , callback ),
or ( aggregatedValue , element , key , container , callback ), where aggregatedValue is the current reduced value,
element is the current element, key is the current key (the current index if container is an Array,
or the current property's name if container is an object), container is the original container,
and callback is the completion's callback.
Each element is processed one at a time, in series.
Calling .parallel() on this async.Plan has no effect, it will process jobs one at a time anyway.
If the iterator fails for one element, the whole process aborts and fails.
If you do *NOT* provide a default aggregatedValue in the async.Plan, then the .exec() method require an initial aggregatedValue as its first argument.
Note that async.reduce( initialAggregatedValue , container , iterator ) is equal to
async.do( container ).iterator( iterator ).aggregator( true , true , initialAggregatedValue ).
Example:
var myArray = [ 'my' , 'wonderful' , 'result' ] ;
var plan = async.reduce( myArray , function( aggregate , element , callback ) {
setTimeout( function() {
// Asyncly calculate the sum of the length
callback( null , aggregate + element.length ) ;
} , 0 ) ;
} )
// No aggregatedValue is provided in the async.Plan creation,
// so the first argument of exec() must be the initial aggregatedValue.
.exec( 0 , function( error , results ) {
// we expect results to be equal to 17
expect( results ).to.be.eql( 17 ) ;
} ) ;
Function( error , results , logicCallback ) triggered for checking if we have to continue or not, where:
mixed any truthy means errorArray or Object that maps the jobsListFunction( [error] , loopAgain ) where:
mixed any truthy means errorBoolean anything else is considered either truthy or falsyArray or ObjectIt performs an async while loop. This is equivalent to javascript code:
while ( expression ) {
// do something
}
Unlike others factories, in order to mimic native language syntax, this factory accepts a whileCallback
rather than a job's list.
So you have to use the async.Plan's .do() method to pass the job's list.
Async while loops behave diffently than other async.Plan in various way:
async.Plan do, but:Example:
async.while( function( error , results , logicCallback ) {
// If doMoreWorksFunction() triggers its callback demanding another loop...
logicCallback( results.moreWorks[ 1 ] === 'loop' ) ;
} )
.do( {
preliminaries: doPreliminariesFunction ,
works: doWorksFunction ,
moreWorks: doMoreWorksFunction
} )
.exec( function( error , results ) {
// 'results' contains only the results of the last loop
thingsToDoWhenFinished() ;
} ) ;
Array or ObjectFunction( error , results , logicCallback ) triggered for checking if we have to continue or not, where:
mixed any truthy means errorArray or Object that maps the jobsListFunction( [error] , loopAgain ) where:
mixed any truthy means errorBoolean anything else is considered either truthy or falsyIt performs an async do-while loop.
It works exactly the same as async.while().do(), except that, by default, the whileCallback is triggered at the end of the process rather than at the beginning. This is equivalent to javascript code:
do {
// do something
} while ( expression )
The following factories instanciate async.Plan of the conditional family.
There are few differencies with async.Plan of the do family.
Jobs have three type of outcome: true, false and error.
Jobs should trigger their callback this way: callback( [error] , result ).
In this case, you are not forced to pass the error argument first.
However, if you pass only one argument, it will be assumed to be an error only if it is an instance of Error.
If an error occurs, it will stop processing any new jobs by default. If true or false is the outcome, then it all depends on the type of conditional.
There are two mode: boolean or not.
When boolean mode is used, any non-error outcome are cast to a boolean value.
In non-boolean mode, the final outcome is simply the outcome of the last processed job.
The non-boolean mode is in line with the way javascript handle expression like myVar1 && myVar2
(it will produce myVar1 if myVar1 is falsy, else myVar2).
By default, jobs are performed in series, one at a time. It is possible to parallelize jobs processing, but it can change the final outcome in non-boolean mode, though the truthness of that outcome remains unchanged.
Array or ObjectIt performs an async conditional AND, so it keeps processing jobs as long as the outcome is truthy.
By default, it uses the non-boolean mode, so the final outcome is the outcome of the last job.
Array or ObjectIt performs an async conditional OR, so it keeps processing jobs as long as the outcome is falsy.
By default, it uses the non-boolean mode, so the final outcome is the outcome of the last job.
Array or ObjectIt performs an async conditional AND, so it keeps processing jobs as long as the outcome is truthy.
By default, it uses the boolean mode, so the final outcome is a boolean.
Array or ObjectIt performs an async conditional OR, so it keeps processing jobs as long as the outcome is falsy.
By default, it uses the boolean mode, so the final outcome is a boolean.
We can create nested conditional statement just like in native language. See the following example:
async.if.and( [
ifSomeConditionsAreMetAnd
async.or( [
ifSomeMoreConditionsAreMet
orIfSomeAlternativeConditionsAreMet
] )
] )
.then( function() {
// Do something if the async conditional statement is true
} )
.else( function() {
// Do something if the async conditional statement is false
} )
.exec() ;
ifSomeConditionsAreMetAnd, ifSomeMoreConditionsAreMet and orIfSomeAlternativeConditionsAreMet
are user functions asyncly checking if some conditions are met or not.
This works because if a job is an instance of async.Plan, the .exec() method will be used as a callback.
We can use as many nested async conditional as we want.
Each factory come with a default set of behaviour. Almost all behaviours can be modified by methods.
However, modifier methods have no effect as soon as an .exec() family method is used on the current async.Plan.
Array or ObjectIt set the job's list. Most of time, the job's list is already passed as the first argument of a factory, so we don't have to use this method.
However, it is used in the async.while().do() scheme, to mimic common programming language syntax.
Number, if omited or true: Infinity, if false: 1It set the parallel limit or concurrency limit. This is the number of async jobs that can be running/pending at a time.
Using a parallel limit value of 1, jobs are processed one at a time, like async.series() factory does.
Using a parallel limit value of Infinity, jobs are processed all at once (if they are async),
like async.parallel() factory does.
Using a parellel limit value of 3, for example, the first three jobs will start at once, when one jobs triggers its callback the fourth job starts, when another job triggers its callback then the fifth job starts, and so on...
Boolean, if omited: trueSet the race mode.
In race mode, the whole jobs processing aborts when the first job finish without error.
See async.race() factory.
Boolean, if omited: trueSet the waterfall mode.
In waterfall mode, each job is called with the previous job output as arguments,
and the first job receives arguments directly from .exec().
See async.waterfall() factory.
Function( error , results , logicCallback ) triggered for checking if we have to continue or not, where:
mixed any truthy means errorArray or Object that maps the jobsListFunction( [error] , loopAgain ) where:
mixed any truthy means errorBoolean anything else is considered either truthy or falsyBoolean, if omited: falseSet a while loop mode.
The argument whileActionBefore is used to define if the condition should be evaluated at the begining of the loop or at the end of the loop.
See async.while().do() (if whileActionBefore is true) or async.do().while() (if whileActionBefore is false) for details.
NumberSet loop mode, the job's list will run n times.
Actually this is a shortcut, it simply set up a while loop with a trivial callback. Avoid to reinvent the wheel again and again.
See .while() for details.
Boolean, if omitted: trueIf errors are fatal (the default in most factories), then whenever a job fails the whole process is aborted immediately.
If error are not fatal, others jobs will be processed even if some errors occurs.
Boolean, if omitted: trueThis only have effects in Conditional family async.Plan.
If castToBoolean is true, the outcome of jobs and the final outcome is always true or false:
this is what happens with async.if.and() and async.if.or() factories by default.
If castToBoolean is false, the outcome of each job remains unchanged, and the final outcome is
the outcome of the last job: this is what happens with async.and() and async.or() factories by default.
Boolean, if omitted: trueThis only have effects in waterfall mode, using async.waterfall() factory.
If transmit is true, each job received the error argument of the previous job.
If transmit is false, the error argument pass by the previous job is not transmitted.
Example with .transmitError:
async.waterfall( [
function( str , callback ) {
// str equals 'oh', passed by .exec()'s first argument
callback( null , str + ' my' ) ;
} ,
function( lastError , str , callback ) {
// lastError equals null
// str equals 'oh my', passed by the previous job
callback( new Error() , str + ' wonderful' ) ;
} ,
function( lastError , str , callback ) {
// lastError is now an instance of Error
// str equals 'oh my wonderful', passed by the previous job
callback( null , str + ' result' ) ;
}
] )
.transmitError( true )
.fatal( false )
.exec( 'oh' , function( error , results ) {
// output 'oh my wonderful result'
console.log( results ) ;
} ) ;
undefined or Number (in ms), if omited: undefinedSet up a time limit for each job.
If a job doesn't trigger its callback within this time, its callback is triggered anyway automatically with an error:
new Error( 'Timeout' ).
If the job triggers its callback later, it will be ignored.
It comes in handy in any network or service dependant async jobs, like database queries, HTTP request, and so on.
Also this is IMPORTANT to understand that this is the async-kit lib who is responsible for the timeout to kick in: the user code is still in execution, it may be pending, waiting for I/O to perform some other tasks. The timeout feature give us the chance to be sure that our callback get triggered within some time limit, it doesn't interupt the job in any way.
Number, it doesn't update if omitedNumber in ms, it doesn't update if omitedNumber, it doesn't update if omitedNumber, in ms, it doesn't update if omitedThis modifier allows jobs in error to be retried.
This is a very nice feature when dealing with other servers or external services, because they could be unavailable at any time, but we don't want important tasks to fail.
It allows fine tuning:
For example, assuming maxRetry: 6, baseTimeout: 100, multiply: 1.5, maxTimeout: 500, we will get for each retry
the timeout value:
A good practice is to specify a low baseTimeout, around 10ms, and a high multiply value, at least 2. This way, things keep reactive when a sporadic error occurs, but if something is really wrong with some of our servers, we didn't flood them to death, we give them a chance to recover.
If maxRetry is high, we may consider using a maxTimeout value, between 10 seconds and 2 minutes. This could be really bad if some actions are retried few hours or few days later, totally out of context.
By the way, those are general guidance, it all depends on the criticy of the tasks, wheither it involves local, lan, vlan or internet networks, and more importantly: if those actions take place behind the scene or if some end-user are currently expecting results quickly.
Example, with some behind the scene cron-like tasks, involving third-party services:
async.parallel( [
retrieveSomeRSS ,
querySomeThirdPartyAPI ,
queryMoreThirdPartyAPI
] )
// At most 100 retries, starting with a 100 ms timeout before retrying,
// multiplying timeout by 2 at each new try but capped at 10 minutes timeout
.retry( 100 , 100 , 2 , 60000 )
.exec( function( error , results ) {
// update your local database or cache
} ) ;
Mixing .timeout() and .retry() can be extremely powerful.
Sometime a task can end up pending a long time, because some bugs occurs, but a retry can eventually succeed immediately: probably we sent a request on some third-party, we get load-balanced to a server that do not respond anymore, but issuing a new request may end up to a server that still works well.
This is exactly what can achieve a mix of .timeout() and .retry(): when the timeout is reached for a job,
it triggers its callback with a failed status (new Error( 'Timeout' )), then retry kick in and the job start over,
it may hit the time limit again and be restarted again, until it succeeds or the retry countdown abort the whole process.
Also there are IMPORTANT drawback we need to be aware of:
.timeout() for details)As a rule of thumb, if we plan to mix .timeout() and .retry(), we must isolate as much as possible critical code,
creating more jobs that perform small task is better.
For example, this is a *VERY* bad practice:
async.do( [
queryMultipleExternalServicesAndThenUpdateOurLocalDatabaseAccordingly
] )
.timeout( 100 )
.retry( 100 , 100 , 2 , 60000 )
.exec( function( error , results ) {
console.log( 'Done!' ) ;
} ) ;
We have to consider rewriting it this way:
async.parallel( [
queryExternalService1 ,
queryExternalService2 ,
queryExternalService3
] )
.timeout( 100 )
.retry( 100 , 100 , 2 , 60000 )
.exec( function( error , results ) {
if ( ! error ) {
updateOurLocalDatabaseAccordingly( results ) ;
}
} ) ;
In the last snippet, we have isolated jobs that can timeout due to things that are out of our control.
If one query failed, we don't have to restart from scratch, re-doing queries that have already succeeded.
Finally, moving updateOurLocalDatabaseAccordingly() into the finallyCallback
of .exec() allows us to use the parallel mode, so the whole process perform faster.
If we had chosen to put this function into a job, we would have been constrained to use an async.series() factory.
More important: we are sure that the code that update our database will run once.
boolean, if omited: trueIf set to true, only the last job pass its result to finallyCallback,
thenCallback etc...
Without .lastJobOnly() (the default in most factories):
async.series( [
function( callback ) { callback( null , 'my' ) ; } ,
function( callback ) { callback( null , 'wonderful' ) ; } ,
function( callback ) { callback( null , 'result' ) ; }
] )
.exec( function( error , result ) {
// result equals `[ [ null , 'my' ], [ null , 'wonderful' ], [ null , 'result' ] ]`
} ) ;
With .lastJobOnly() (default in async.waterfall() and async.race() factories):
async.series( [
function( callback ) { callback( null , 'my' ) ; } ,
function( callback ) { callback( null , 'wonderful' ) ; } ,
function( callback ) { callback( null , 'result' ) ; }
] )
.lastJobOnly()
.exec( function( error , result ) {
// result equals `'result'`
} ) ;
BE CAREFUL: when using .lastJobOnly() in parallel mode, this is the job that finish last which transmits its results.
This is *NOT* necessarly the last job in the job's list.
Note that .lastJobOnly() is used in async.race() factory, but here the whole process abort when the first job finish
without error, so the first job and the last job are the same.
Boolean, if omited: trueIf set to true, the results directly map the jobsList.
It is used (and locked) in async.map() factory.
If set to false, the results contains for each entry, the whole argument's list
passed by the job's callback.
Without .mapping1to1() (the default in most factories):
async.parallel( [
function( callback ) { callback( null , 'my' ) ; } ,
function( callback ) { callback( null , 'wonderful' ) ; } ,
function( callback ) { callback( null , 'result' ) ; }
] )
.exec( function( error , results ) {
// results equals `[ [ null , 'my' ], [ null , 'wonderful' ], [ null , 'result' ] ]`
} ) ;
With .mapping1to1() (the default in async.map() factory):
async.map( [
function( callback ) { callback( null , 'my' ) ; } ,
function( callback ) { callback( null , 'wonderful' ) ; } ,
function( callback ) { callback( null , 'result' , 'extra argument that will be dropped' ) ; }
] )
.exec( function( error , results ) {
// results equals `[ 'my' , 'wonderful' , 'result' ]`
} ) ;
Note: when using .mapping1to1(), any extra arguments passed to the job's callback are ignored.
Function, Array or ObjectArgument passed to .using() is used in combination with the job's list.
Behaviours all depend on the type of the arguments.
In the following .using() variation, async.do() can be replaced by any async.Plan's factory.
Array (or Object) of ArrayFunctionWhen combining .do() and .using() this way, each job contains an array of arguments to pass to workerFunction.
Example:
async.do( [
[ 'http://example.com/' , 500 ] ,
[ 'http://example.com/forum/' , 800 ] ,
[ 'http://example.com/blog/' , 200 ]
] )
.using( function( url , timeout ) {
// Async check of url, with some timeout
} )
.exec( function( error , results ) {
if ( ! error ) { console.log( "Success!" ) ; }
} ) ;
Also, if your workerFunction only accepts one argument, you can avoid Array of Array construct:
async.do( [
'http://example.com/' ,
'http://example.com/forum/' ,
'http://example.com/blog/'
] )
.using( function( url ) {
// Async check of url
} )
.exec( function( error , results ) {
if ( ! error ) { console.log( "Success!" ) ; }
} ) ;
Array (or Object) of FunctionArrayThis is the opposite. Here we have a list of different function, but they take the same arguments.
Example:
async.do( [
dnsResolve ,
ping ,
httpGet
] )
.using( 'http://example.com/' )
.exec( function( error , results ) {
if ( ! error ) { console.log( "Success!" ) ; }
} ) ;
In the previous snippet, .using() provide the data, and .do() provide the actions, where dnsResolve, ping
and httpGet are three functions that take an URL as their first arguments. The dnsResolve function will convert
the URL into an IP addresse, then ping will er... ping this IP, and finally httpGet will forge an HTTP request
and get the page content.
Function( element , [key] , [container] , callback ) where:
mixed the current array element or object's property valueNumber or String the current key (index for array, property name for object)Array or Object, this is the original containerFunction( error , [arg1] , [arg2] , ... ) a node-style callback to trigger on completionWith .iterator( iteratorFunction ) our jobs become data for iteratorFunction.
This is close to the behaviour of .using( workerFunction ), except that an iterator function is not called the same way.
Rather than processing each element of the Array as an array of arguments, here the whole element is passed as the
first argument of the iterator.
In fact, async.do( container ).iterator( iteratorFunction ) is equal to async.foreach( container , iteratorFunction ).
See async.foreach() for details.
Boolean, if omited: trueBoolean, if omited: truemixed, this is the default valueThis set or unset the current async.Plan as an aggregator.
Note that async.do( container ).iterator( iterator ).aggregator( true , true , initialAggregatedValue )
is equal to async.reduce( initialAggregatedValue , container , iterator ).
For more details, see async.reduce().
If transmitAggregate is set, then the iterator (or job's function) receive the current aggregatedValue as its first argument, all other arguments being shifted to the right.
If returnAggregate is set, then the results passed to callback (then, catch and finally callback) only contains the aggregatedValue.
If defaultAggregate is set, this is what will be used as the starting value for aggregatedValue.
Number, any number is ok but recommended values are between -20 and +20This try to mimic the unix command nice and renice.
This set up how the job's scheduler behaves.
It depends on the niceness value:
setImmediate() internally. This prevent from the Maximum call stack size exceeded error when callbacks are synchronous,
and give some breath for I/O when dealing with CPU-bound tasks. As long as things are synchronous, there will be no
difference between async.series() or async.parallel().setImmediate() internally. This scheduling allows I/O to be performed
(see setImmediate() for details).setTimeout() internally. This scheduling allows I/O to be performed
and much more. The niceness value is used as the delay for setTimeout(), so using .nice(10)
means that the scheduler will delay further action for 10ms
(see setTimeout() for details).See NextGen Event nice feature for references.
By default, if .nice() is not called, the nice value is -20 (i.e. synchronous for at most 19 recursions).
Full synchronous scheduling may cause Maximum call stack size exceeded issues if loop, .retry() or just an huge job's
list is involved, because everything use nested callback the way we would have done it, those nested callback are just
abstracted away by the lib, but still remains behind the scene.
That's why starting at v0.6.0, there isn't full synchronous scheduling anymore: once in a while, an asynchronous call
will be triggered. Do not drop the nice value below -20, which provide at most 19 recursions.
Asynchronous scheduling uses the javascript's event loop, so there is no more infinite nested callback possible.
It can scale better for big job's list, loop and .retry()...
If we have a big synchronous task to do, we can divide it into many jobs, then use for example:
async.series( jobsList ).nice( 0 ).exec() ;
... to asyncify it a bit. This can be very important for services: our application must keep accepting new request during the big task processing. Also if the task is really that big, it is usually a good practice to spawn a process or create a new specific service for this particular task anyway.
Function( results )
mixed, depends on optionsThis set up a then callback part of the async.Plan itself.
See thenCallback for details.
Function( results )
mixed, depends on optionsThis set up an else callback part of the async.Plan itself.
See elseCallback for details.
This has no effect for Do family async.Plan.
Function( error , results )
mixed, depends on jobs' codemixed, depends on optionsThis set up a catch callback part of the async.Plan itself.
See catchCallback for details.
Function( error , results )
mixed, depends on jobs' codemixed, depends on optionsThis set up a finally callback part of the async.Plan itself.
See finallyCallback for details.
This method is used to clone an async.Plan and return it.
The cloned async.Plan is unlocked: we can use its modifier methods even if the original async.Plan is locked
or is currently under execution.
String, one of 'exec', 'execKV', 'execFinally', 'execThenCatch', 'execThenElse', 'execThenElseCatch'
and 'execArgs'... if omited: 'exec'This export and return an async.Plan as a function.
By default, the exported function behaves exactly like the .exec() method of the async.Plan.
If we want to export a different .exec()-like method, we can provide the method's name as the argument of .export().
Since the async.Plan is internally cloned, changes made on the original async.Plan do not change how the exported function behaves.
This method execute the async.Plan.
Until an exec-like method is called, nothing happens at all, previous methods mostly configure the async.Plan.
Arguments passed to .exec() depend on factories by default, and can be modified by .execMapping().
However, most factories use this scheme:
.exec( [arg1] , [arg2] , ... , [finallyCallback](#ref.callback.finallyCallback) ).
mixed : arguments to pass to all the jobs (or to the first job only in waterfall mode)Function( error , results )
mixed, depends on jobs' codemixed, depends on optionsFollowing .exec()-like methods have a static scheme, and are not modified by .execMapping().
Function( error , results )
mixed, depends on jobs' codemixed, depends on optionsThis method execute the async.Plan, just like .exec().
It only accepts one argument: the finallyCallback.
Function( results )
mixed, depends on optionsFunction( error , results )
mixed, depends on jobs' codemixed, depends on optionsFunction( error , results )
mixed, depends on jobs' codemixed, depends on optionsThis method execute the async.Plan, just like .exec().
Like the name suggests, the first argument should be the thenCallback, and
catchCallback as the second.
However, the finallyCallback can still be passed as the third argument.
Function( results )
mixed, depends on optionsFunction( results )
mixed, depends on optionsFunction( error , results )
mixed, depends on jobs' codemixed, depends on optionsThis method execute the async.Plan, just like .exec().
Like the name suggests, the first argument should be the thenCallback, and
elseCallback as the second.
However, the finallyCallback can still be passed as the third argument.
Function( results )
mixed, depends on optionsFunction( results )
mixed, depends on optionsFunction( error , results )
mixed, depends on jobs' codemixed, depends on optionsFunction( error , results )
mixed, depends on jobs' codemixed, depends on optionsThis method execute the async.Plan, just like .exec().
Like the name suggests, the first argument should be the thenCallback,
elseCallback as the second, and catchCallback as the third.
However, the finallyCallback can still be passed as the fourth argument.
mixedThis method execute the async.Plan, just like .exec().
All arguments passed to this method are passed to all the jobs (except in waterfall mode, where they are passed only to the first job).
Object
Boolean, if omited: falseNumber (integer), if omited: 0Number (integer), if omited: 0Array of String describing each input (only used for function signature), if omited: []Array of String (can only be: 'then', 'else', 'catch' and 'finally'), if omited: []This method is used to configure .exec()'s behaviour.
If config.aggregateArg is true, the first argument of .exec() is the aggregate's value.
If config.maxInputs is greater than 0, the next arguments of .exec() *MAY* be inputs for jobs (arguments passed to them).
If config.minInputs is greater than 0, the next arguments of .exec() *MUST* be inputs for jobs.
In fact, .exec() supports variable number of arguments.
Note that in waterfall mode, inputs arguments are only passed to the first job.
Finally, if config.callbacks is not an empty array, the last arguments are callback, strictly in the order defined.
.exec() supports variable number of arguments:
if config.minInputs and config.maxInputs are equals, the number of inputs arguments are fixed,
so the number of callback is variable: some callback could be omited
if config.minInputs and config.maxInputs are *NOT* equals, the number of inputs arguments are variable,
so the number of callback is fixed (if it wasn't, we couldn't have a clue weither an argument is an input or a callback)
Example using the async.Plan property .execMappingSignature to get the signature of .exec(), here with variable number of inputs:
var plan = async.do( [
// Some jobs
] )
.execMapping( {
callbacks: [ 'then' , 'catch' ] ,
minInputs: 0 ,
maxInputs: 2 ,
inputsName: [ 'firstArg' , 'secondArg' ]
} ) ;
console.log( plan.execMappingSignature ) ;
// produce: ( [firstArg], [secondArg], thenCallback, catchCallback )
Example with fixed number of inputs:
var plan = async.do( [
// Some jobs
] )
.execMapping( {
callbacks: [ 'then' , 'catch' ] ,
minInputs: 2 ,
maxInputs: 2 ,
inputsName: [ 'firstArg' , 'secondArg' ]
} ) ;
console.log( plan.execMappingSignature ) ;
// produce: ( firstArg, secondArg, [thenCallback], [catchCallback] )
Example with config.aggregateArg set to true:
var plan = async.do( [
// Some jobs
] )
.execMapping( {
aggregateArg: true ,
callbacks: [ 'then' , 'catch' ] ,
minInputs: 2 ,
maxInputs: 2 ,
inputsName: [ 'firstArg' , 'secondArg' ]
} ) ;
console.log( plan.execMappingSignature ) ;
// produce: ( aggregateValue, firstArg, secondArg, [thenCallback], [catchCallback] )
Object
Array input arguments for jobs, if omited: []mixed optionnal aggregate initial valueFunction optionnal thenCallbackFunction optionnal elseCallbackFunction optionnal catchCallbackFunction optionnal finallyCallbackThis method execute the async.Plan, just like .exec().
Rather than passing arguments in a predefined order, .execKV() accepts an object of key-value pairs.
This is an alternative to .execMapping() & .exec().
Pro:
.exec()Cons:
.execMapping(), .exec() can raise error if misused, for example it constraints a number of input's argumentsThose callbacks are triggered (if conditions are met) when the async.Plan is resolved.
Note that if we don't use .timeout() and a job is pending forever, the async.Plan will never being resolved,
thus no callback will be ever triggered.
There are two stages of callback.
The first stage are callbacks defined in the async.Plan itself. Those callback are *ALWAYS* triggered before the second stage.
The second stage are callbacks of the .exec()-like methods.
Function( results )
mixed, depends on optionsFor Do family, this callback is triggered if the async.Plan's execution succeed. The success depends on factory and options used.
Usually, an async.Plan succeed if no error happened. But jobs on error can be retried if .retry() is used, and finally succeed,
async.race succeed as long as one job succeed, and so on.
Furthermore, for Conditional family, the final result should be true or truthy for this callback to be triggered.
The results argument's format passed to this callback depends on many factor. See related factories and modifier.
Function( results )
mixed, depends on optionsIt never triggers for Do family async.Plan.
For Conditional family, it will trigger if the final result is false or falsy.
However, if no catchCallback exists for this stage (see callbacks introduction for what a callback stage is),
it will trigger if the final outcome is an error too.
Function( error , results )
mixed, depends on jobs' codemixed, depends on optionsThis callback is triggered when the final outcome is an error.
Function( error , results )
mixed, depends on jobs' codemixed, depends on optionsThis callback is *ALWAYS* triggered. This is the last callback of a stage to be triggered.
Function( error , results , logicCallback ), where:
mixed any truthy means errorArray or Object that maps the jobsListFunction( [error] , loopAgain ) where:
mixed any truthy means errorBoolean anything else is considered either truthy or falsyThis callback is used for while loop.
The last iteration's error and results are passed to this function.
Then the internal logicCallback function can be triggered, if a truthy value is passed as the loopAgain argument, a new loop iteration will be performed, if a falsy value is passed, no new loop iteration will take place: completion callback (thenCallback, elseCallback, catchCallback, finallyCallback) will be triggered depending on the current (last) iteration's outcome.
An instance of async.ExecContext is returned by each exec()-like methods.
We can use this object to listen to some useful event.
This method provide insightful real-time information about the status of each jobs. This is designed for flow-control debugging/logging purpose, other uses are discouraged.
It returns an Object or an Array that map the jobs' list.
For each job, an object is given where:
mixed the original job, e.g. Function, Array, async.Plan, etc...string the current status of the job, one of the following:
Array the result of the job, if any... it is strictly the same format that exec()-like function
pass to their callbacksArray a list of errors that happened for this job, e.g. each multiple failure on retryable jobs, as well
as reporting when the job has called its completion callback multiple times, and so oninteger the number of time the job has been tried (useful in conjunction with .retry()).Object, with properties:
Number the number of resolved jobs (done/error/aborted)Number the number of resolved jobs that have succeededNumber the number of resolved jobs that have failedNumber the number of jobs started and still running (i.e. not done)Number the number of jobs in queue, not started yetNumber the loop iteration (while loop or .repeat())mixed the current error status, Conditional family async.Plan DO NOT pass this argumentArray of mixed for Do family async.Plan or just mixed for Conditional family async.Plan, this is the partial resultsThe 'progress' event is fired each time a job complete.
The progressStatus object contains the main informations necessary to build a progress bar.
Others arguments can be useful if we need access to the partial results.
mixed the current error status, Conditional family async.Plan DO NOT pass this argumentArray of mixed for Do family async.Plan or just mixed for Conditional family async.Plan,
this is the final resultsThe 'resolved' event is fired when the final result is settled.
This event triggers thenCallback(), elseCallback(), catchCallback() and finallyCallback().
If we listen to this event, the above callbacks will always trigger first (since they have already registered).
So there is only few cases where it is useful to listen to it.
Sometime it can be useful to register for this event directly in jobs (using this which references the current
async.ExecContext instance), so we can abort a CPU consuming job that will be ignored anyway.
When in concurrency with others, the 'resolved' event is always fired before any others events.
mixed the current error status, Conditional family async.Plan DO NOT pass this argumentArray of mixed for Do family async.Plan or just mixed for Conditional family async.Plan, this is NOT
the final (i.e. resolved) results: jobs that finish after the 'resolved' event will have their results listed too, so this
can be different from what we get from the 'resolved' event.The 'finish' event is fired after the 'resolved' event, when all remaining running jobs are finished.
In series flow, there is practically no differences with the 'resolved' event.
However, in a parallel flow, many jobs are running at the same time, if one job finish with an error, the final result is settled right now,
so the 'resolved' event is fired, however all other pending jobs have to be done for the 'finish' event to be fired.
Alternatively, when using async.race, the first non-error job to finish settle the final result and fire
the 'resolved' event, so the 'finish' event is fired when all racing jobs are done.
Most of time, this event is not so useful, however there are cases where we do not want to continue until nothing run in the background anymore.
When in concurrency with others, the 'finish' event is always fired after any others events.
Job's function, using function and iterator function automatically get an async.JobContext instance as its this context. We can use this object to perform some particular task.
This immutable property directly point to the current async.ExecContext's instance.
So you can use it to listen to event directly from within the job, for example.
Calling this.abort() from inside a job immediately aborts the current job's queue, and triggers completion callbacks.
Arguments passed works the same way than regular callback( [error] , [arg1] , [arg2], [...] ).
In fact, in most cases, this is the same than callback( new Error( 'Error!' ) , arg1, arg2, [...] ) except that it will
abort the job's queue even when a regular error wouldn't.
That's it, even if the async.Plan as been created with .fatal( false ), or we have set .retry(), or even if the
error parameter is falsy, it will abort anyway.
This can be useful if a job succeed, but require that nothing else should be run afterward.
Notice: An async while loop will *NOT* be aborted: *ONLY* the current loop iteration will be aborted, the whileAction will be called immediately to evaluate if it should loop again or not.
Notice: It has no effect on Conditional family async.Plan.
This event is triggered if the current job has been timed out by the underlying lib.
This can happen when using the .timeout() method of an async.Plan instance.
number the exit codenumber the maximum time allowed for each underlying listener before aborting, default to 1000 (ms).This is a replacement for process.exit(), that is async-friendly.
When process.exit() is called, the 'exit' event is emited on the process object, and each listener can perform a last
synchronous task before the whole process exit. Sadly, a lot of things in Node.js are working asynchronously, and thus cannot
be handled properly that way.
But thanks to async.exit() those old days are gone!
When you call async.exit(), it emits the 'asyncExit' event on the process object.
There are two kinds of listeners:
function( [code] , [timeout] ) listeners, that does not have a callback, are interested in the event but they don't need
to perform critical tasks or they can handle critical tasks synchronously. E.g.: a server that will not accept connection
or data anymore after receiving this event.
function( code , timeout , completionCallback ) listeners, that do have a callback, have some critical async tasks to perform
before exiting. E.g.: a server that needs to gracefully exit will not accept connection or data anymore, but it still
has to finish client request still in progress.
Note that code and timeout arguments passed to listeners are actual values processed by async.exit().
So async.exit() will simply wait for all listeners having a completionCallback to trigger it (or being timed out) before exiting.
Example:
process.on( 'asyncExit' , function( code , timeout , callback ) {
console.log( 'asyncExit event received - starting a short task' ) ;
setTimeout( function() {
console.log( 'Short task finished' ) ;
callback() ;
} , 100 ) ;
} ) ;
process.on( 'asyncExit' , function( code , timeout ) {
console.log( 'asyncExit event received - non-critical task' ) ;
setTimeout( function() {
console.log( 'Non-critical task finished' ) ;
} , 200 ) ;
} ) ;
async.exit( 5 , 500 ) ;
After 100ms, it will produce:
asyncExit event received - starting a short task
asyncExit event received - non-critical task
Short task finished
Note how the setTimeout's function is not executed in the second event handler: this handler does not accept a callback,
hence the process will exit as soon as the first handler is done: after 100ms.
function the function to execute after the timeoutnumber the time before running the functionIt returns an opaque value representing the timer.
This is almost the same than setTimeout().
This is great for true timeout function, like this:
function fn( callback_ )
{
var called = false ;
var callback = function() {
if ( called ) { return ; }
called = true ;
callback.apply( this , arguments ) ;
}
doSomethingAsync( callback ) ;
async.setSafeTimeout( callback.bind( new Error( 'Time out!' , 100 ) ) ) ;
}
What if CPU-bound/synchronous task should be done after calling fn(), e.g. loading synchronously a lot of configuration files?
If performing those tasks takes more than 100ms, a regular setTimeout() would trigger the callback after 100ms whatever happened,
but instead async.setSafeTimeout() try to give to doSomethingAsync() 100ms to perform its duty.
In other words: the timeout start after the event loop take the control back.
It also try to not timeout something that has finished its job, but has its callback queued after the timeout callback.
Like clearTimeout() but for timer created using async.setSafeTimeout().
Note that it can clear setTimeout() timers too.
The Mocha framework is used for BDD-style tests.
The Full BDD spec generated by Mocha can found here.
FAQs
A simple and powerful async abstraction lib for easily writing Node.js code.
The npm package async-kit receives a total of 1,490 weekly downloads. As such, async-kit popularity was classified as popular.
We found that async-kit demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Socket and Seal Security collaborate to fix a critical npm overrides bug, resolving a three-year security issue in the JavaScript ecosystem's most popular package manager.
Security News
TypeScript is porting its compiler to Go, delivering 10x faster builds, lower memory usage, and improved editor performance for a smoother developer experience.
Research
Security News
The Socket Research Team has discovered six new malicious npm packages linked to North Korea’s Lazarus Group, designed to steal credentials and deploy backdoors.