lerna-parallelism-load-balanced

lerna-parallelism

lerna-parallelism is an extension of lerna that adds support for parallel execution / splitting, for e.g. CircleCI's parallelism feature.

Installation

Project-local Installation

yarn add -D lerna-parallelism

# And then run via:
yarn lerna-parallelism ...

Global Installation

yarn global add lerna-parallelism
# or
volta install lerna-parallelism

# And then run via:
lerna-parallelism ...

Basic Usage

lerna-parallelism adds two additional CLI options on top of lerna:

• --split n: The number of split partitions. Defaults to $CIRCLE_NODE_TOTAL.
• --partition n: Which partition to execute, zero-based. Defaults $CIRCLE_NODE_INDEX.

All other command options behave just like the upstream lerna version of the respective command.

The following commands from lerna are supported:

lerna-parallelism run

lerna run

For instance, this executes the last of four partitions. It also passes along --stream & --concurrency 1 to prefix log lines with the package name.

yarn lerna run \
  --stream \
  --concurrency 1 \
  --split 4 \
  --partition 3 \
  test

Other Commands

For some commands, like lerna bootstrap, splitting makes no sense. For some others, it does, specifically:

• lerna changed
• lerna exec
• lerna list
• lerna publish

If you'd like to see support for these commands as well, feel free to submit a pull request!

Load Balancing (advanced)

By default, lerna-parallelism gives each partition an approximately equal number of packages by chunking an alphabetized array. Depending on the number of tests each of your packages has, you may find this results in imbalanced runtimes between partitions.

To address this, lerna-parallelism offers a load balancing mode which uses a weighted round-robin algorithm, informed by a weight "hint" specified in each package's package.json.

This adds two additional CLI options in addition to those above:

• --loadBalance: Toggles load-balancing mode. Defaults to false.
• --packageTestWeightKey myProjectWeight: The lookup key used to read the project's weight from its package.json. Defaults to lernaPackageTestWeight.

Usage

lerna-parallelism --loadBalance run test

And add a lernaPackageTestWeight property (numeric) to your package.json for each package. If it is missing, 1 is the default weight assigned.

If you have a need to partition differently for multiple CI tasks, you can use --packageTestWeightKey to specify which weight property should be read from package.json.

License

This project is licensed under the ISC License.