ez-streams - npm Package Compare versions

Comparing version 0.1.3 to 0.1.4

API.md

@@ -12,3 +12,5 @@ # ez-streams
 * [ez-streams/lib/devices/file](lib/devices/file.md)  
-   File based ES streams
+   File based EZ streams
+* [ez-streams/lib/devices/galaxy](lib/devices/galaxy.md)  
+   Stream constructors for galaxy
 * [ez-streams/lib/devices/generic](lib/devices/generic.md)  
@@ -28,2 +30,4 @@ Generic stream constructors
    EZ wrappers for oracle
+* [ez-streams/lib/devices/sqlserver](lib/devices/sqlserver.md)  
+   EZ wrappers for oracle
 * [ez-streams/lib/devices/std](lib/devices/std.md)  
@@ -33,2 +37,6 @@ EZ wrappers for standard I/O streams
    In-memory string streams
+* [ez-streams/lib/devices/uturn](lib/devices/uturn.md)  
+   Special device that transforms a writer into a reader
+* [ez-streams/lib/mappers/convert](lib/mappers/convert.md)  
+   Encoding mappers
 * [ez-streams/lib/reader](lib/reader.md)  
@@ -35,0 +43,0 @@ EZ Streams core reader API

lib/devices/file.md

@@ -1,2 +0,2 @@
-## File based ES streams
+## File based EZ streams
 
@@ -15,1 +15,11 @@ `var ez = require('ez-streams');`
   creates an EZ writer that writes to a binary file.    
+* `reader = ez.devices.file.list(path, options)`  
+  `reader = ez.devices.file.list(path, recurse, accept)`  
+  creates a reader that enumerates (recursively) directories and files.  
+  Returns the entries as `{ path: path, name: name, depth: depth, stat: stat }` objects.  
+  Two `options` may be specified: `recurse` and `accept`.  
+  If `recurse` is falsy, only the entries immediately under `path` are returned.  
+  If `recurse` is truthy, entries at all levels (including the root entry) are returned.  
+  If `recurse` is `"postorder"`, directories are returned after their children.  
+  `accept` is an optional function which will be called as `accept(_, entry)` and 
+  will control whether files or subdirectories will be included in the stream or not.  

lib/index.js

@@ -20,3 +20,4 @@ "use strict";
 	devices: requireDir('devices'),
+	mappers: requireDir('mappers'),
 	transforms: requireDir('transforms'),
 };

lib/reader.md

@@ -85,2 +85,8 @@ ## EZ Streams core reader API
   Returns a stream which is identical to the original one but in which up to `max` entries may have been buffered.  
+* `stream = reader.nodify()`  
+  converts the reader into a native node Readable stream.  
+* `reader = reader.nodeTransform(duplex)`  
+  pipes the reader into a node duplex stream. Returns another reader. 
+* `cmp = reader1.compare(_, reader2)`  
+  compares reader1 and reader2 return 0 if equal,  
 ## StreamGroup API
@@ -87,0 +93,0 @@ * `reader = group.dequeue()`

lib/writer.md

@@ -11,5 +11,3 @@ ## EZ Streams core writer API
   Returns `proto` for convenience.
-* `writer = writer.premap(fn, thisObj)`  
-  Similar to `map` on arrays.  
-  The `fn` function is called as `fn(_, elt, i)`.  
-  Returns another writer on which other operations may be chained.
+* `stream = writer.nodify()`  
+  converts the writer into a native node Writable stream.  

package.json

 {
   "name": "ez-streams",
   "description": "EZ streams for node.js",
-  "version": "0.1.3",
+  "version": "0.1.4",
   "repository": {
@@ -6,0 +6,0 @@ "type": "git",

README.md

@@ -11,2 +11,5 @@ # Easy Streams for node.js
 
+EZ streams are also compatible with [galaxy](https://github.com/Sage/galaxy). See [Galaxy-support](#galaxy-support) below for details.
+
+<a name="installation"/>
 ## Installation
@@ -18,2 +21,3 @@
 
+<a name="devices"/>
 ## Creating a stream
@@ -28,3 +32,3 @@
 var textRd = ez.devices.file.text.reader(path); // text file reader
-var binWr = ez.devices.file.text.writer(path); // binary file writer
+var binWr = ez.devices.file.binary.writer(path); // binary file writer
 var stringRd = ez.devices.string.reader(text); // in memory text reader
@@ -78,2 +82,3 @@ ```
 
+<a name="basic-api"/>
 ## Basic read and write
@@ -93,2 +98,3 @@
 
+<a name="array-api"/>
 ## Array-like API
@@ -143,2 +149,3 @@
 
+<a name="pipe"/>
 ## Pipe
@@ -172,2 +179,3 @@
 
+<a name="infinite-streams"/>
 ## Infinite streams
@@ -199,2 +207,3 @@
 
+<a name="transforms"/>
 ## Transformations
@@ -253,2 +262,3 @@
 
+<a name="transforms-library"/>
 ## Transforms library
@@ -275,2 +285,33 @@
 
+<a name="node-interop"/>
+## Interoperabily with native node.js streams
+
+`ez-streams` are fully interoperable with native node.js streams.
+
+You can convert a node.js stream to an _ez_ stream:
+
+``` javascript
+// converting a node.js readable stream to an ez reader
+var reader = ez.devices.node.reader(stream);
+// converting a node.js writable stream to an ez writer
+var writer = ez.devices.node.writer(stream);
+```
+
+You can also convert in the reverse direction, from an _ez_ stream to a node.js stream:
+
+``` javascript
+// converting an ez reader to a node readable stream
+var stream = reader.nodfiy();
+// converting an ez writer to a node writable stream
+var stream = writer.nodify();
+```
+
+And you can transform an _ez_ stream with a node duplex stream:
+
+``` javascript
+// transforms an ez reader into another ez reader
+reader = reader.nodeTransform(duplexStream)
+```
+
+<a name="lookahead"/>
 ## Lookahead
@@ -288,2 +329,3 @@
 
+<a name="parallelizing"/>
 ## Parallelizing
@@ -303,2 +345,3 @@
 
+<a name="fork-and-join"/>
 ## Fork and join
@@ -332,2 +375,3 @@
 
+<a name="exceptions"/>
 ## Exception handling
@@ -360,2 +404,21 @@
 
+<a name="writer-chaining"/>
+## Writer chaining
+
+You can also chain operations on writers via a special `pre` property. For example:
+
+``` javascript
+// create a binary file writer
+var rawWriter = ez.devices.file.binary.writer("data.gzip");
+// create another writer that applies a gzip transform before the file writer
+var zipWriter = rawWriter.pre.nodeTransform(zlib.createGzip());
+```
+
+All the chainable operations available on readers (`map`, `filter`, `transform`, `nodeTransform`, ...) 
+can also be applied to writers through this `pre` property.
+
+Note: the `pre` property was introduced to stress the fact that the operation is applied _before_ 
+writing to the original writer, even though it appears _after_ in the chain.
+
+<a name="backpressure"/>
 ## Backpressure
@@ -371,2 +434,41 @@
 
+<a name="galaxy-support"/>
+## Galaxy support
+
+EZ streams is the recommended streams package for [galaxy](https://github.com/Sage/galaxy). The API is overloaded to facilitate integration with generator functions. 
+
+If you develop with galaxy, you should use the API as follows:
+
+* add a `Star` postfix to the _reducer_ methods (the methods that have `_` as first parameter: `forEach`, `every`, `some`, `reduce`, `pipe`, `toArray`). You should also `yield` on these `Star` calls.
+* pass generator functions (`function*(...) { ... }`) instead of regular asynchronous functions (`function(_, ...) { ...}`) to the methods that expect a callback (`forEach`, `map`, `filter`, `transform`, ...).
+
+For example, instead of:
+
+``` javascript
+console.log("pi~=" + 4 * numberReader(10000).filter(function(_, n) {
+	return n % 2; // keep only odd numbers
+}).map(function(_, n) {
+	return n % 4 === 1 ? 1 / n : -1 / n;
+}).reduce(_, function(_, res, val) {
+	return res + val;
+}, 0));
+```
+
+you would write:
+
+``` javascript
+console.log("pi~=" + 4 * (yield numberReader(10000).filter(function*(n) {
+	return n % 2; // keep only odd numbers
+}).map(function*(n) {
+	return n % 4 === 1 ? 1 / n : -1 / n;
+}).reduceStar(function*(res, val) {
+	return res + val;
+}, 0)));
+```
+
+_Note:_ do not forget the `*` after `function` in the functions inside your chains; and do not forget to `yield` on reducers (`reduceStar` in the example above).
+
+See the [galaxy unit test](test/server/galaxy-test.js) for more examples.
+
+<a name="api"/>
 ## API
@@ -376,2 +478,3 @@
 
+<a name="more-info"/>
 ## More information
@@ -383,4 +486,5 @@
 
+<a name="license"/>
 # License
 
 This work is licensed under the terms of the [MIT license](http://en.wikipedia.org/wiki/MIT_License).

New alerts

Major refactor

Supply chain risk

Package has recently undergone a major refactor. It may be unstable or indicate significant internal changes. Use caution when updating to versions that include significant changes.

Found 1 instance in 1 package

Uses eval

Supply chain risk

Package uses dynamic code execution (e.g., eval()), which is a dangerous practice. This can prevent the code from running in certain environments and increases the risk that the code may contain exploits or malicious behavior.

Found 1 instance in 1 package

Improved metrics

Total package byte prevSize

160932

increased by21.03%

Number of package files

76

increased by20.63%

Lines of code

407

increased by578.33%

Number of lines in readme file

474

increased by28.11%

Worsened metrics

Number of medium supply chain risk alerts

2

increased byInfinity%

No dependency changes