mississippi
Advanced tools
Comparing version 1.2.0 to 1.3.0
{ | ||
"name": "mississippi", | ||
"version": "1.2.0", | ||
"version": "1.3.0", | ||
"description": "a collection of useful streams", | ||
@@ -17,2 +17,3 @@ "main": "index.js", | ||
"from2": "^2.1.0", | ||
"parallel-transform": "^1.1.0", | ||
"pump": "^1.0.0", | ||
@@ -19,0 +20,0 @@ "pumpify": "^1.3.3", |
@@ -24,2 +24,3 @@ # mississippi | ||
- [finished](#finished) | ||
- [parallel](#parallel) | ||
@@ -32,3 +33,3 @@ ### pipe | ||
When using standard `source.pipe(destination)` the source will _not_ be destroyed if the destination emits close or error. You are also not able to provide a callback to tell when then pipe has finished. | ||
When using standard `source.pipe(destination)` the source will _not_ be destroyed if the destination emits close or error. You are also not able to provide a callback to tell when the pipe has finished. | ||
@@ -39,3 +40,3 @@ `miss.pipe` does these two things for you, ensuring you handle stream errors 100% of the time (unhandled errors are probably the most common bug in most node streams code) | ||
`miss.pipe` is provided by [`require('pump')`](https://npmjs.org/pump) | ||
`miss.pipe` is provided by [`require('pump')`](https://www.npmjs.com/package/pump) | ||
@@ -62,3 +63,3 @@ #### example | ||
Iterate the data in `stream` one chunk at a time. Your `each` function will be called with with `(data, next)` where data is a data chunk and next is a callback. Call `next` when you are ready to consume the next chunk. | ||
Iterate the data in `stream` one chunk at a time. Your `each` function will be called with `(data, next)` where data is a data chunk and next is a callback. Call `next` when you are ready to consume the next chunk. | ||
@@ -69,3 +70,3 @@ Optionally you can call `next` with an error to destroy the stream. You can also pass the optional third argument, `done`, which is a function that will be called with `(err)` when the stream ends. The `err` argument will be populated with an error if the stream emitted an error. | ||
`miss.each` is provided by [`require('stream-each')`](https://npmjs.org/stream-each) | ||
`miss.each` is provided by [`require('stream-each')`](https://www.npmjs.com/package/stream-each) | ||
@@ -105,3 +106,3 @@ #### example | ||
`miss.pipeline` is provided by [`require('pumpify')`](https://npmjs.org/pumpify) | ||
`miss.pipeline` is provided by [`require('pumpify')`](https://www.npmjs.com/package/pumpify) | ||
@@ -145,3 +146,3 @@ #### example | ||
`miss.duplex` is provided by [`require('duplexify')`](https://npmjs.org/duplexify) | ||
`miss.duplex` is provided by [`require('duplexify')`](https://www.npmjs.com/package/duplexify) | ||
@@ -175,3 +176,3 @@ #### example | ||
`miss.through` is provided by [`require('through2')`](https://npmjs.org/through2) | ||
`miss.through` is provided by [`require('through2')`](https://www.npmjs.com/package/through2) | ||
@@ -189,6 +190,6 @@ #### example | ||
function (chunk, enc, cb) { | ||
cb(chunk.toString().toUpperCase()) | ||
cb(null, chunk.toString().toUpperCase()) | ||
}, | ||
function (cb) { | ||
cb('ONE LAST BIT OF UPPERCASE') | ||
cb(null, 'ONE LAST BIT OF UPPERCASE') | ||
} | ||
@@ -218,3 +219,3 @@ ) | ||
`miss.from` is provided by [`require('from2')`](https://npmjs.org/from2) | ||
`miss.from` is provided by [`require('from2')`](https://www.npmjs.com/package/from2) | ||
@@ -265,3 +266,3 @@ #### example | ||
`miss.to` is provided by [`require('flush-write-stream')`](https://npmjs.org/flush-write-stream) | ||
`miss.to` is provided by [`require('flush-write-stream')`](https://www.npmjs.com/package/flush-write-stream) | ||
@@ -308,3 +309,3 @@ #### example | ||
Calling `miss.concat(cb)` returns a writable stream. `cb` is called when the writable stream is finished, e.g. when all data is done being written to it. `cb` is called with a single argument, `(data)`, which will containe the result of concatenating all the data written to the stream. | ||
Calling `miss.concat(cb)` returns a writable stream. `cb` is called when the writable stream is finished, e.g. when all data is done being written to it. `cb` is called with a single argument, `(data)`, which will contain the result of concatenating all the data written to the stream. | ||
@@ -315,3 +316,3 @@ Note that `miss.concat` will not handle stream errors for you. To handle errors, use `miss.pipe` or handle the `error` event manually. | ||
`miss.concat` is provided by [`require('concat-stream')`](https://npmjs.org/concat-stream) | ||
`miss.concat` is provided by [`require('concat-stream')`](https://www.npmjs.com/package/concat-stream) | ||
@@ -351,3 +352,3 @@ #### example | ||
`miss.finished` is provided by [`require('end-of-stream')`](https://npmjs.org/end-of-stream) | ||
`miss.finished` is provided by [`require('end-of-stream')`](https://www.npmjs.com/package/end-of-stream) | ||
@@ -368,1 +369,41 @@ #### example | ||
### parallel | ||
#####`miss.parallel(concurrency, each)` | ||
This works like `through` except you can process items in parallel, while still preserving the original input order. | ||
This is handy if you wanna take advantage of node's async I/O and process streams of items in batches. With this module you can build your very own streaming parallel job queue. | ||
Note that `miss.parallel` preserves input ordering, if you don't need that then you can use [through2-concurrent](https://github.com/almost/through2-concurrent) instead, which is very similar to this otherwise. | ||
#### original module | ||
`miss.parallel` is provided by [`require('parallel-transform')`](https://npmjs.org/parallel-transform) | ||
#### example | ||
This example fetches the GET HTTP headers for a stream of input URLs 5 at a time in parallel. | ||
```js | ||
function getResponse (item, cb) { | ||
var r = request(item.url) | ||
r.on('error', function (err) { | ||
cb(err) | ||
}) | ||
r.on('response', function (re) { | ||
cb(null, {url: item.url, date: new Date(), status: re.statusCode, headers: re.headers}) | ||
r.abort() | ||
}) | ||
} | ||
miss.pipe( | ||
fs.createReadStream('./urls.txt'), // one url per line | ||
split(), | ||
miss.parallel(5, getResponse), | ||
miss.through(function (row, enc, next) { | ||
console.log(JSON.stringify(row)) | ||
next() | ||
}) | ||
) | ||
``` |
License Policy Violation
LicenseThis package is not allowed per your license policy. Review the package's license to ensure compliance.
Found 1 instance in 1 package
License Policy Violation
LicenseThis package is not allowed per your license policy. Review the package's license to ensure compliance.
Found 1 instance in 1 package
14013
395
10
+ Addedparallel-transform@^1.1.0
+ Addedcyclist@1.0.2(transitive)
+ Addedparallel-transform@1.2.0(transitive)