Security News
NIST Misses 2024 Deadline to Clear NVD Backlog
NIST has failed to meet its self-imposed deadline of clearing the NVD's backlog by the end of the fiscal year. Meanwhile, CVE's awaiting analysis have increased by 33% since June.
Builder takes your npm
tasks and makes them composable, controllable from
a single point, and flexible.
npm
is fantastic for controlling dependencies, tasks (via scripts
) and
general project workflows. But a project-specific package.json
simply doesn't
scale when you're managing many (say 5-50) very similar repositories.
Enter Builder. Builder is "almost" npm
, but provides for off-the-shelf
"archetypes" to provide central sets of package.json
scripts
,
dependencies
and devDependencies
. The rest of this page will dive into
the details and machinations of the tool, but first here are a few of the
rough goals and motivations behind the project.
npm
workflow. So much so, that we include a section in this
guide on how to abandon the use of Builder in a project and revert everything
from archetypes back to vanilla npm
package.json
scripts
, dependencies
and devDependencies
.At a high level builder
is a tool for consuming package.json
scripts
commands, providing sensible / flexible defaults, and supporting various scenarios
("archetypes") for your common use cases across multiple projects.
Builder is not opinionated, although archetypes are and typically dictate file structure, standard configurations, and dev workflows. Builder supports this in an agnostic way, providing essentially the following:
NODE_PATH
, PATH
enhancements to run, build, import from archetypes so
dependencies and configurations don't have to be installed directly in a
root project.run
) or multiple concurrent tasks
(concurrent
).package.json
scripts
tasks.... and that's about it!
To start using builder, install and save builder
and any archetypes you
intend to use. We'll use the builder-react-component archetype as an
example.
Note: Most archetypes have an ARCHETYPE
package and parallel
ARCHETYPE-dev
npm package. The ARCHETYPE
package contains almost
everything needed for the archetype (prod dependencies, scripts, etc.) except
for the devDependencies
which the latter ARCHETYPE-dev
package is solely
responsible for bringing in.
For ease of use, one option is to globally install builder
and locally install
archetypes:
$ npm install -g builder
$ npm install --save builder-react-component
$ npm install --save-dev builder-react-component-dev
Like a global install of any Node.js meta / task runner tool (e.g., eslint
,
mocha
, gulp
, grunt
) doing a global install is painful because:
... so instead, we strongly recommend a local install described in the next section!
To help you keep up with project-specific builder requirements, a globally-installed
builder
will detect if a locally-installed version of builder
is
available and switch to that instead:
$ /GLOBAL/PATH/TO/builder
[builder:local-detect] Switched to local builder at: ./node_modules/builder/bin/builder-core.js
... now using local builder! ...
To avoid tying yourself to a single, global version of builder
, the option
that we endorse is locally installing both builder
and archetypes:
$ npm install --save builder
$ npm install --save builder-react-component
$ npm install --save-dev builder-react-component-dev
However, to call builder
from the command line you will either need to
augment your PATH
variable with a shell configuration (Mac/Linux) like:
export PATH="./node_modules/.bin:${PATH}"
# ... OR ...
export PATH="${PATH}:./node_modules/.bin"
or call the longer ./node_modules/.bin/builder
instead of builder
from the
command line.
After builder
is available, you can edit .builderrc
like:
---
archetypes:
- builder-react-component
to bind archetypes.
... and from here you are set for builder
-controlled meta goodness!
Display general or command-specific help (which shows available specific flags).
$ builder help
$ builder help <action>
Run builder help <action>
for all available options. For a quick overview:
Run a single task from script
. Analogous to npm run <task>
$ builder run <task>
Flags:
--builderrc
: Path to builder config file (default: .builderrc
)--tries
: Number of times to attempt a task (default: 1
)--setup
: Single task to run for the entirety of <action>
.Run multiple tasks from script
concurrently. Roughly analogous to
npm run <task1> | npm run <task2> | npm run <task3>
, but kills all processes on
first non-zero exit (which makes it suitable for test tasks), unless --no-bail
is provided.
$ builder concurrent <task1> <task2> <task3>
Flags:
--builderrc
: Path to builder config file (default: .builderrc
)--tries
: Number of times to attempt a task (default: 1
)--setup
: Single task to run for the entirety of <action>
.--queue
: Number of concurrent processes to run (default: unlimited - 0|null
)--[no-]buffer
: Buffer output until process end (default: false
)--[no-]bail
: End all processes after the first failure (default: true
)Note that tries
will retry individual tasks that are part of the concurrent
group, not the group itself. So, if builder concurrent --tries=3 foo bar baz
is run and bar fails twice, then only bar
would be retried. foo
and baz
would only execute once if successful.
Run a single task from script
concurrently for each item in an array of different
environment variables. Roughly analogous to:
$ FOO=VAL1 npm run <task> | FOO=VAL2 npm run <task> | FOO=VAL3 npm run <task>
... but kills all processes on first non-zero exit (which makes it suitable for
test tasks), unless --no-bail
is provided. Usage:
$ builder envs <task> <json-array>
$ builder envs <task> --envs-path=<path-to-json-file>
Examples:
$ builder envs <task> '[{ "FOO": "VAL1" }, { "FOO": "VAL2" }, { "ENV1": "VAL3" }]'
$ builder envs <task> '[{ "FOO": "VAL1", "BAR": "VAL2" }, { "FOO": "VAL3" }]'
Flags:
--builderrc
: Path to builder config file (default: .builderrc
)--tries
: Number of times to attempt a task (default: 1
)--setup
: Single task to run for the entirety of <action>
.--queue
: Number of concurrent processes to run (default: unlimited - 0|null
)--[no-]buffer
: Buffer output until process end (default: false
)--[no-]bail
: End all processes after the first failure (default: true
)--envs-path
: Path to JSON env variable array file (default: null
)Just like npm run <task> [-- <args>...]
,
flags after a --
token in a builder task or from the command line are passed
on to the underlying tasks. This is slightly more complicated for builder in
that composed tasks pass on the flags all the way down. So, for tasks like:
"scripts": {
"down": "echo down",
"way": "builder run down -- --way",
"the": "builder run way -- --the",
"all": "builder run the -- --all"
}
We can run some basics (alone and with a user-added flag):
$ builder run down
down
$ builder run down -- --my-custom-flag
down --my-custom-flag
If we run the composed commands, the --
flags are accumulated:
$ builder run all
down --way --the --all
$ builder run all -- --my-custom-flag
down --way --the --all --my-custom-flag
The rough heuristic here is if we have custom arguments:
builder <action>
command, append with --
to pass through.builder
command, then append without --
token.The underlying concept here is that builder
script
commands simply are
npm-friendly package.json
script
commands. Pretty much anything that you
can execute with npm run <task>
can be executed with builder run <task>
.
Builder can run 1+ tasks based out of package.json
scripts
. For a basic
scenario like:
{
"scripts": {
"foo": "echo FOO",
"bar": "echo BAR"
}
}
Builder can run these tasks individually:
$ builder run foo
$ builder run bar
Sequentially via ||
or &&
shell helpers:
$ builder run foo && builder run bar
Concurrently via the Builder built-in concurrent
command:
$ builder concurrent foo bar
With concurrent
, all tasks continue running until they all complete or
any task exits with a non-zero exit code, in which case all still alive tasks
are killed and the Builder process exits with the error code.
Archetypes deal with common scenarios for your projects. Like:
Archetypes typically provide:
package.json
with builder
-friendly script
tasks.script
tasks.In most cases, you won't need to override anything. But, if you do, pick the
most granular scripts
command in the archetype you need to override and
define just that in your project's package.json
script
section. Copy
any configuration files that you need to tweak and re-define the command.
The easiest bet is to just have one archetype per project. But, multiple are
supported. In terms of scripts
tasks, we end up with the following example:
ROOT/package.json
ROOT/node_modules/ARCHETYPE_ONE/package.json
ROOT/node_modules/ARCHETYPE_TWO/package.json
Say we have a .builderrc
like:
---
archetypes:
- ARCHETYPE_ONE
- ARCHETYPE_TWO
The resolution order for a script
task (say, foo
) present in all three
package.json
's would be the following:
ROOT/package.json
then the configured archetypes in reverse
order: ARCHETYPE_TWO/package.json
, then ARCHETYPE_ONE/package.json
for
a matching task foo
foo
, check if it is a "passthrough" task, which means it delegates
to a later instance -- basically "foo": "builder run foo"
. If so, then look
to next instance of task found in order above.Archetypes use conventional scripts
task names, except for the following
special cases:
"npm:postinstall"
"npm:preversion"
"npm:version"
"npm:test"
These tasks are specifically actionable during the npm
lifecycle, and
consequently, the archetype mostly ignores those for installation by default,
offering them up for actual use in your project.
As an additional restriction, non-npm:FOO
-prefixed tasks with the same
name (e.g., FOO
) may call then npm:
-prefixed task, but not the other
way around. So
// Good / OK
"npm:test": "builder run test-frontend",
"test": "builder run npm:test",
// Bad
"npm:test": "builder run test",
"test": "builder run test-frontend",
Builder uses some magic to enhance NODE_PATH
to look in the root of your
project (normal) and in the installed modules of builder archetypes. This
latter path enhancement sometimes throws tools / libraries for a loop. We
recommend using require.resolve("LIBRARY_OR_REQUIRE_PATH")
to get the
appropriate installed file path to a dependency.
This comes up in situations including:
The other thing that comes up in our Archetype configuration file is the
general requirement that builder is running from the project root, not
relative to an archetype. However, some libraries / tools will interpret
"./"
as relative to the configuration file which may be in an archetype.
So, for these instances and instances where you typically use __dirname
,
an archetype may need to use process.cwd()
and be constrained to only
ever running from the project root. Some scenarios where the process.cwd()
path base is necessary include:
require.resolve
-ed)The execution of tasks generally must originate from Builder, because of all
of the environment enhancements it adds. So, for things that themselves exec
or spawn processes, like concurrently
, this can be a problem. Typically, you
will need to have the actual command line processes invoked by Builder.
Builder uses exec
under the hood with piped stdout
and stderr
. Programs
typically interpret the piped environment as "doesn't support color" and
disable color. Consequently, you typically need to set a "force color"
option on your executables in scripts
commands if they exist.
So, why exec
and not spawn
or something similar that has a lot more process
control and flexibility? The answer lies in the fact that most of what Builder
consumes is shell strings to execute, like script --foo --bar "Hi there"
.
Parsing these arguments into something easily consumable by spawn
and always
correct is quite challenging. exec
works easily with straight strings, and
since that is the target of scripts
commands, that is what we use for Builder.
Builder is designed to be as close to vanilla npm as possible. So, if for
example you were using the builder-react-component
archetype with a project
package.json
like:
"scripts": {
"postinstall": "builder run npm:postinstall",
"preversion": "builder run npm:preversion",
"version": "builder run npm:version",
"test": "builder run npm:test",
/* other deps */
},
"dependencies": {
"builder": "v2.0.0",
"builder-react-component": "v0.0.5",
/* other deps */
},
"devDependencies": {
"builder-react-component-dev": "v0.0.5",
/* other deps */
}
and decided to no longer use Builder, here is a rough set of steps to unpack the archetype into your project and remove all Builder dependencies:
ARCHETYPE/package.json:dependencies
to your
PROJECT/package.json:dependencies
(e.g., from builder-react-component
).
You do not need to copy over ARCHETYPE/package.json:devDependencies
.ARCHETYPE/package.json:scripts
to your
PROJECT/package.json:scripts
that do not begin with the builder:
prefix.
Remove the npm:
prefix from any scripts
tasks and note that you may have
to manually resolve tasks of the same name within the archetype and also with
your project.ARCHETYPE-dev/package.json:dependencies
to your
PROJECT/package.json:devDependencies
(e.g., from builder-react-component-dev
)ARCHETYPE
into the root project.
For example, for builder-react-component
you would need to copy the
builder-react-component/config
directory to PROJECT/config
(or a renamed
directory).scripts
tasks and:
builder run <task>
with npm run <task>
builder concurrent <task1> <task2>
tasks, first install the
concurrently
package and then rewrite to:
concurrent 'npm run <task1>' 'npm run <task2>'
... and (with assuredly a few minor hiccups) that's about it! You are
Builder-free and back to a normal npm
-controlled project.
The builder
project effectively starts at v2.x.x
. Prior to that Builder was
a small DOM utility that fell into disuse, so we repurposed it for a new
wonderful destiny! But, because we follow semver, that means everything starts
at v2
and as a helpful tip / warning:
Treat
v2.x
as av0.x
release
We'll try hard to keep it tight, but at our current velocity there are likely
to be some bumps and API changes that won't adhere strictly to semver until
things settle down in v3.x
-on.
2.4.0
builder <action> <task> [-- <args>...]
support.
builder-react-component#27FAQs
An NPM-based task runner
The npm package builder receives a total of 1,092 weekly downloads. As such, builder popularity was classified as popular.
We found that builder demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 32 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
NIST has failed to meet its self-imposed deadline of clearing the NVD's backlog by the end of the fiscal year. Meanwhile, CVE's awaiting analysis have increased by 33% since June.
Security News
Cloudflare has launched a setup wizard allowing users to easily create and manage a security.txt file for vulnerability disclosure on their websites.
Security News
The Socket Research team breaks down a malicious npm package targeting the legitimate DOMPurify library. It uses obfuscated code to hide that it is exfiltrating browser and crypto wallet data.