import { exec } from "@transform/process";
import { concat, lines } from "@transform/stream";
import { emitItems, interleave, skipLast } from "@transform/core";

await expect(
  pipeline(
    emitItems("Hello", "beautiful", "world!"),
    interleave("\n"),
    exec(`sed 's/^/> /g' | grep -v beautiful`),
    concat()
  ),
  "to yield items",
  ["> Hello\n> world!\n"]
);

You can also pipe data into a sub-process.

await expect(
  pipeline(
    emitItems("Hello\nfantastic\nworld"),
    exec("grep -v fantastic"),
    lines()
  ),
  "to yield items",
  ["Hello", "world", ""]
);

spawn

Spawns a new sub-process. See Node child_process.spawn for more information.

import { spawn } from "@transform/process";
import { lines } from "@transform/stream";
import { pipeline, skipLast } from "@transform/core";

You can use it to emit items into the pipeline.

await expect(
  pipeline(spawn("ls", [testDir]), lines(), skipLast()),
  "to yield items",
  ["0.txt", "1.txt", "2.txt"]
);

You can also pipe data into a sub-process.

await expect(
  pipeline(
    emitItems("Hello\nfantastic\nworld"),
    spawn("grep", ["-v", "fantastic"]),
    lines(),
    skipLast()
  ),
  "to yield items",
  ["Hello", "world"]
);

Multiple sub-processes can be combined.

await expect(
  pipeline(spawn("ls", [testDir]), spawn("grep", ["0"]), lines(), skipLast()),
  "to yield items",
  ["0.txt"]
);

startProcess/childProcess

Starts a child process pipeline in a new Node instance.

const { startProcess, childProcess } = require("@transformation/process");
import { pipeline } from "@transform/core";

Notice this is only useful for cases where your pipeline is more CPU intensive than the overhead of communicating with the child process.

As an example let's try to square numbers in a child process.

We start by defining the child process pipeline (square.js).

module.exports = childProcess(map((n) => n * n));

Now we can load start the process as part of our pipeline.

await expect(
  pipeline(
    emitItems(0, 1, 2, 3, 4, 5, 6, 7, 8, 9),
    startProcess("square.js"))
  ),
  "to yield items",
  [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
);

What is happening here, is that every item is serialized and send into the child process for processing. When the child process emits new items, they are serialized and passed back to the main pipeline.

Keywords

transformation,automation,pipeline,process,io

FAQs

What is @transformation/process?

Is @transformation/process popular?

Is @transformation/process well maintained?

Package last updated on 17 Jan 2022

Did you know?

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

@transformation/process