builder - npm Package Compare versions

Comparing version 2.10.1 to 3.0.0-beta.1

CONTRIBUTING.md

 Contributing
 ============
 
+## Pull Requests
+
+Before submitting a PR / publishing a release make sure to:
+
+* Run all tests / checks with: `npm run builder:check`
+* Update and commit changes to README.md with: `npm run builder:build`
+
 Thanks for helping out!
 
-## Checks, Tests
+## Contributor Covenant Code of Conduct
 
-Run `npm run builder:check`
+### Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+### Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+### Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+### Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+### Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at [e-mail]. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+### Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

HISTORY.md

 History
 =======
 
+## Unreleased
+
+* **BREAKING**: Remove `--unlimited-buffer` command line option. It is unneeded
+  with switch to `spawn`.
+* Switch from `exec` to `spawn` child process spawning for better stdio/stderr
+  propagation. ( [@exogen][] )
+  [#20](https://github.com/FormidableLabs/builder/issues/20)
+* Add auto-TOC to README.md.
+  [#101](https://github.com/FormidableLabs/builder/issues/101)
+
 ## 2.10.1
@@ -5,0 +15,0 @@

lib/args.js

@@ -74,7 +74,2 @@ "use strict";
       default: "info"
-    },
-    "unlimited-buffer": {
-      desc: "Use an unlimited buffer for shell commands",
-      types: [Boolean],
-      default: false
     }
@@ -81,0 +76,0 @@ },

lib/environment.js

@@ -54,4 +54,4 @@ "use strict";
  * Resolution order:
- * 1. `ROOT/node_modules/.bin`
- * 2. `ROOT/node_modules/ARCHETYPE[1-n]/node_modules/.bin`
+ * 1. `ROOT/node_modules/ARCHETYPE[1-n]/node_modules/.bin`
+ * 2. `ROOT/node_modules/.bin`
  * 3. existing `PATH`
@@ -78,4 +78,4 @@ *
  * Resolution order:
- * 1. `ROOT/node_modules/`
- * 2. `ROOT/node_modules/ARCHETYPE[1-n]/node_modules/`
+ * 1. `ROOT/node_modules/ARCHETYPE[1-n]/node_modules/`
+ * 2. `ROOT/node_modules/`
  * 3. existing `NODE_PATH`
@@ -82,0 +82,0 @@ *

lib/runner.js

 "use strict";
-/*eslint max-params: [2, 4]*/
+/*eslint max-params: [2, 4], max-statements: [2, 20] */
 
-var exec = require("child_process").exec;
 var path = require("path");
@@ -9,2 +8,3 @@ var _ = require("lodash");
 var chalk = require("chalk");
+var spawn = require("./spawn");
 var log = require("./log");
@@ -17,16 +17,2 @@ var Tracker = require("./utils/tracker");
 
-// One limitation of `exec()` is that it unconditionally buffers stdout/stderr
-// input (whether piped, listened, or whatever) leading to a `maxBuffer` bug:
-// https://github.com/FormidableLabs/builder/issues/62
-//
-// We set a comfortable margin here to up the number. In the future, we could
-// just go "whole hog" and bump to `Infinity` if needed.
-//
-// Longer term, we can consider whether we want to do what npm does and use
-// `spawn` with manual OS-compatible `sh` vs. `cmd` detection, cobble together
-// our own flags and manage everything so that we can use the much more flexible
-// `spawn` instead of `exec`.
-// https://github.com/FormidableLabs/builder/issues/20
-var MAX_BUFFER = 32 * 1024 * 1024;
-
 // Helper for command strings for logging.
@@ -162,11 +148,23 @@ var cmdStr = function (cmd, opts) {
 var run = function (cmd, shOpts, opts, callback) {
-  var maxBuffer = opts.unlimitedBuffer ? Infinity : MAX_BUFFER;
+  // Buffer output (for concurrent usage).
+  var buffer = !!opts.buffer;
+  var bufs = []; // { type: `stdio|stderr`, data: `data` }
 
+  // Copied from npm's lib/utils/lifecycle.js
+  var sh = "sh";
+  var shFlag = "-c";
+
   // Update shell options and ensure basic structure.
   shOpts = _.extend({
-    maxBuffer: maxBuffer,
-    env: {}
+    env: {},
+    stdio: buffer ? "pipe" : "inherit"
   }, shOpts);
 
-  var buffer = opts.buffer;
+  // Copied from npm's lib/utils/lifecycle.js
+  if (process.platform === "win32") {
+    sh = process.env.comspec || "cmd";
+    shFlag = "/d /s /c";
+    shOpts.windowsVerbatimArguments = true;
+  }
+
   var env = shOpts.env;
@@ -181,12 +179,12 @@
 
-
   log.info("proc:start", cmdStr(cmd, opts));
-  var proc = exec(cmd, shOpts, function (err, stdout, stderr) {
+  var proc = spawn(sh, [shFlag, cmd], shOpts, function (err) {
     var code = err ? err.code || 1 : 0;
     var level = code === 0 ? "info" : "warn";
 
-    // Write out buffered output.
+    // Output buffered output.
     if (buffer) {
-      process.stdout.write(stdout);
-      process.stderr.write(stderr);
+      bufs.forEach(function (buf) {
+        process[buf.type].write(buf.data.toString());
+      });
     }
@@ -198,6 +196,10 @@
 
-  // Concurrent / "whenever" output.
-  if (!buffer) {
-    proc.stdout.pipe(process.stdout, { end: false });
-    proc.stderr.pipe(process.stderr, { end: false });
+  // Gather buffered output in memory.
+  if (buffer) {
+    proc.stdout.on("data", function (data) {
+      bufs.push({ type: "stdout", data: data });
+    });
+    proc.stderr.on("data", function (data) {
+      bufs.push({ type: "stderr", data: data });
+    });
   }
@@ -204,0 +206,0 @@

package.json

 {
   "name": "builder",
-  "version": "2.10.1",
+  "version": "3.0.0-beta.1",
   "description": "An NPM-based task runner",
@@ -16,2 +16,4 @@ "repository": {
   "scripts": {
+    "builder:build-md-toc": "doctoc --notitle README.md",
+    "builder:build": "npm run builder:build-md-toc",
     "builder:lint-server": "eslint --color -c .eslintrc-server lib bin",
@@ -39,2 +41,3 @@ "builder:lint-server-test": "eslint --color -c .eslintrc-server-test test",
     "coveralls": "^2.11.4",
+    "doctoc": "^1.2.0",
     "eslint": "^1.7.3",
@@ -41,0 +44,0 @@ "eslint-config-defaults": "^7.0.1",

README.md

@@ -44,2 +44,58 @@ [![Travis Status][trav_img]][trav_site]
 
+**Contents**:
+
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+
+
+- [Overview](#overview)
+  - [Usage](#usage)
+    - [Global Install](#global-install)
+    - [Local Install](#local-install)
+      - [PATH Augmentation](#path-augmentation)
+      - [Full Path Invocation](#full-path-invocation)
+    - [Configuration](#configuration)
+    - [Builder Actions](#builder-actions)
+      - [builder run](#builder-run)
+      - [builder concurrent](#builder-concurrent)
+      - [builder envs](#builder-envs)
+        - [Custom Flags](#custom-flags)
+        - [Expanding the Archetype Path](#expanding-the-archetype-path)
+- [Tasks](#tasks)
+- [npm Config](#npm-config)
+  - [`npm` Config Overview](#npm-config-overview)
+  - [Builder Configs](#builder-configs)
+  - [Config Notes](#config-notes)
+    - [Tip - Use String Values](#tip---use-string-values)
+    - [npmrc Configuration](#npmrc-configuration)
+    - [Command Line Environment Variables](#command-line-environment-variables)
+- [Archetypes](#archetypes)
+  - [Task Resolution](#task-resolution)
+  - [Special Archetype Tasks](#special-archetype-tasks)
+  - [Creating an Archetype](#creating-an-archetype)
+    - [Initializing a Project](#initializing-a-project)
+    - [Managing the `dev` Archetype](#managing-the-dev-archetype)
+    - [Node Require Resolution and Module Pattern](#node-require-resolution-and-module-pattern)
+      - [The Module Pattern](#the-module-pattern)
+        - [ES.next Imports and The Module Pattern](#esnext-imports-and-the-module-pattern)
+    - [Frontend Resolution and Module Pattern](#frontend-resolution-and-module-pattern)
+    - [Application vs. Archetype Dependencies](#application-vs-archetype-dependencies)
+    - [Moving `dependencies` and `scripts` to a New Archetype](#moving-dependencies-and-scripts-to-a-new-archetype)
+      - [Moving `dependencies` and `devDependencies` from an Existing `package.json`](#moving-dependencies-and-devdependencies-from-an-existing-packagejson)
+      - [Moving `scripts` and Config Files](#moving-scripts-and-config-files)
+      - [Updating Path and Module References in Config Files](#updating-path-and-module-references-in-config-files)
+    - [Example `builder` Archetype Project Structure](#example-builder-archetype-project-structure)
+- [Tips, Tricks, & Notes](#tips-tricks-&-notes)
+  - [PATH, NODE_PATH Resolution](#path-node_path-resolution)
+  - [Alternative to `npm link`](#alternative-to-npm-link)
+  - [Project Root](#project-root)
+  - [Avoid npm Lifecycle Commands](#avoid-npm-lifecycle-commands)
+  - [Other Process Execution](#other-process-execution)
+  - [Terminal Color](#terminal-color)
+  - [Why Exec?](#why-exec)
+  - [I Give Up. How Do I Abandon Builder?](#i-give-up-how-do-i-abandon-builder)
+  - [Versions v1, v2, v3](#versions-v1-v2-v3)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
 ## Overview
@@ -227,3 +283,2 @@
 * `--log-level`: Level to log at (`info`, `warn`, `error`, `none`)
-* `--unlimited-buffer`: Unlimited shell output buffer.
 * `--expand-archetype`: Expand `node_modules/<archetype>` with full path (default: `false`)
@@ -252,3 +307,2 @@ * `--builderrc`: Path to builder config file (default: `.builderrc`)
 * `--log-level`: Level to log at (`info`, `warn`, `error`, `none`)
-* `--unlimited-buffer`: Unlimited shell output buffer.
 * `--expand-archetype`: Expand `node_modules/<archetype>` with full path (default: `false`)
@@ -296,3 +350,2 @@ * `--builderrc`: Path to builder config file (default: `.builderrc`)
 * `--log-level`: Level to log at (`info`, `warn`, `error`, `none`)
-* `--unlimited-buffer`: Unlimited shell output buffer.
 * `--expand-archetype`: Expand `node_modules/<archetype>` with full path (default: `false`)
@@ -924,2 +977,57 @@ * `--builderrc`: Path to builder config file (default: `.builderrc`)
 
+#### Frontend Resolution and Module Pattern
+
+An analogous situation occurs for frontend JS code in the production archetype,
+but with a different solution. If frontend JS code has dependencies within a dev
+archetype, the build environment will need to be enhanced to search the
+dev archetype's `node_modules`. (This often occurs in frontend test suites).
+
+For Webpack, this means adding the dev archetype modules directory explicitly
+to the code (`resolve.root`) and loader (`resolveLoader.root`) configurations
+as appropriate. So, something like:
+
+```js
+// <archetype>/config/webpack.config.test.js
+
+// Stash the location of `<archetype-dev>/node_modules`
+//
+// A normal `require.resolve` looks at `package.json:main`. We instead want
+// just the _directory_ of the module. So use heuristic of finding dir of
+// package.json which **must** exist at a predictable location.
+var archetypeDevNodeModules = path.join(
+  path.dirname(require.resolve("<archetype-dev>/package.json")),
+  "node_modules"
+);
+
+// Webpack configuration.
+module.exports = {
+  // ...
+  resolve: {
+    // ...
+    root: [archetypeNodeModules]
+  },
+  resolveLoader: {
+    // ...
+    root: [archetypeNodeModules]
+  }
+};
+```
+
+For other frontend loaders like Browserify, Rollup, etc., an analogous
+configuration would be required.
+
+Note that you should _only_ use this pattern for files that are used for dev
+workflows. For example, if `webpack.config.js` is part of the prod workflow
+(for maybe a `postinstall` build or something), then you can't do a
+`path.dirname(require.resolve("<archetype-dev>/package.json"))` because the dev
+archetype isn't installed. Instead, only add the dev archetype modules directory
+to code that can only be called from **dev** workflows.
+
+**Shared Node / Frontend Code**: Unfortunately, the preferred Node and Webpack
+methods of importing dev archetype dependencies are _different_, which makes
+setup a little awkward for shared code that runs both on the frontend and in
+Node. While this situation won't often come up for dev dependencies, if it does
+one option is to do an environment detect and conditionally do different imports
+based on if in Node or frontend (Webpack).
+
 #### Application vs. Archetype Dependencies
@@ -926,0 +1034,0 @@

New alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

New author

Supply chain risk

A new npm collaborator published a version of the package for the first time. New collaborators are usually benign additions to a project, but do indicate a change to the security surface area of a package.

Found 1 instance in 1 package

Environment variable access

Supply chain risk

Package accesses environment variables, which may be a sign of credential stuffing or data theft.

Found 1 instance in 1 package

No v1

Quality

Package is not semver >=1. This means it is not stable and does not support ^ ranges.

Found 1 instance in 1 package

Fixed alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Improved metrics

Total package byte prevSize

106378

increased by10.61%

Number of package files

16

increased by6.67%

Lines of code

1510

increased by2.86%

Number of lines in readme file

1424

increased by8.21%

Worsened metrics

Dev dependency count

11

increased by10%

Number of low quality alerts

1

increased byInfinity%

Number of low supply chain risk alerts

11

increased by10%

Number of medium supply chain risk alerts

2

increased by100%

No dependency changes