@esy-nightly/esy

esy

package.json workflow for native development with Reason/OCaml

This README serves as a development documentation for esy. For user documentation refer to esy.sh documentation site.

• Repository structure
• Workflow
  • Testing Locally
  • Running Tests
  • Branches
  • Issues
  • Publishing Releases

Repository structure

The following snippet lists esy repository structured (omitting irrelevant or obvious items) with further explanations:

├── CHANGELOG.md
├── LICENSE
├── README.md
│
├── Makefile
│   Common tasks and workflows for esy development.
│
├── bin/esy
│   symlink (wrapper on Windows) for esy command, used for running tests
│
├── bin/esyInstallRelease.js
│   postinstall step for npm releases produced with `esy npm-release`
│   command. This is a built JS file which is developed in a separate flow
│   inside `esy-install-npm-release/` subdirectory (see below).
│
├── docs
│   esy end user documentation in markdown format.
│
├── dune
├── dune-project
│
├── esy
│   This dune library implements sandbox builder - a routine which builds
│   the entire dependency graph and provides other introspection APIs.
│
├── esy/bin
│   This dune executable implements "esy" command.
│
├── esy-solve
│   This dune library implements solver.
│
├── esy-fetch
│   This dune library implements installer - fetching and installing of package sources
│
├── esy-build-package
│   This dune library implements package builder. esy library uses this to
│   build each package.
│
├── esy-build-package/bin
│   This dune executable implements "esy-build-package" command.
│
├── esy-install-npm-release
│   Sources for `bin/esyInstallRelease.js`.
│
├── esy-command-expression
│   Parser for #{...} syntax used in esy manifests.
│
├── esy-shell-expansion
│   A simple shell expansion.
│
├── esy-lib
│   A collection of utility modules shared between other libraries.
│
├── site
│   Sources for https://esy.sh
│
├── esy.lock
│   Lock files. Esy uses itself for development
│
├── package.json
│   Manifest for yarn to manage NPM dependencies of this project
│
├── scripts
│
├── test
│   Unit tests.
│
├── test-e2e-slow
│   End-to-end test suite which takes a significant amount of time since they're 
│   not mocked or rarely so.
│   We execute it on CI by placing `@slowtest` token in commit messages.
│
└── test-e2e
    End-to-end test suite that dont need the network. Heavily mocked

Workflow

To make changes to esy and test them locally:

% git clone  --recurse-submodules git://github.com/esy/esy.git
% cd esy # Change to the cloned directory
% esy # install and build dependencies 

And then run newly built esy executable from anywhere by adding PATH_TO_REPO/_build/install/default/bin to the $PATH during the shell's session. On Windows, append PATH_TO_REPO/bin too.

Updating bin/esyInstallRelease.js

bin/esyInstallRelease.js is developed separately within the esy-install-npm-release/ directory.

Run:

% make bin/esyInstallRelease.js

to update the bin/esyInstallRelease.js file with the latest changed, don't forget to commit it.

Running Tests

esy has primarily 3 kinds of tests.

• Unit tests - useful when developing parsers etc
• Slow end-to-end tests
• Fast end-to-end tests

Unit Tests

These are present inline in the *.re files. To run them,

esy b dune runtest

Fast end-to-end tests

These are present in test-e2e folder and are written in JS. They're run by jest

yarn jest

Slow end-to-end tests

They're present in test-e2e-slow and are written in JS. They're supposed to mimick the user's workflow as closely as possible.

By placing @slowtest token in commit messages, we mark the commit ready for the slow tests framework (tests that hit the network). They are run with node test-e2e-slow/run-slow-tests.js

Windows

In cases e2e tests fail with Host key verification failed., you might have to create ssh keys in the cygwin shall and add them to your github profile.

• Enter cygwin installed by esy (not the global one)

.\node_modules\esy-bash\re\_build\default\bin\EsyBash.exe bash

• Generate ssh keys

ssh-keygen

• Add the public key to you Github profile

• Add the following to the bash rc of the cygwin instance

eval $(ssh-agent -s)
ssh-add ~/.ssh/id_rsa

Branches

There are two branches:

• master — the active development, we cut new versions out of there regularly.
• 0.0.x — maintainance branch for 0.0.x releases.
• 0.2.x — maintainance branch for 0.2.x releases.
• 0.3.x — maintainance branch for 0.3.x releases.

Workflow for esy.sh

To make changes to esy.sh:

• Bootstrap site's dev environment:

% make site-bootstrap

• Run site locally:

% make site-start

• When you are happy with the changes:

% make site-publish

Issues

Issues are tracked at esy/esy.

Publishing Releases

esy is released on npm.

Because esy is written in OCaml/Reason and compiled into a native executable we need to acquire a set of prebuilt binaries for each supported platform (Windows, macOS and Linux). We employ CI servers (thanks Azure) to build platform specific releases.

The release workflow is the following:

• Ensure you are on master branch and assuming you want to release the version currently defined in package.json (see step 6.), run

  % make release-tag
  % git push && git push --tags
  

• Wait till CI finishes its task and release @esy-nightly/esy package.

  You can test it manually.

• Run

  % make release-prepare
  

  which downloads the nightly corresponding to the current commit working directory is at and "promotes" it to a release. It will create _release/package directory.

• Ensure release inside _release/package directory is ok.

  You can cd _release/package && npm pack && npm install -g ./esy-*.tgz to test how release installs and feels.

• Run

  % make release-publish
  

  to upload the release on npm.

  Use

  % make NPM_RELEASE_TAG=next release-publish
  

  to publish the release under next tag (so users won't get it automatically but only explicitly requested).

• Bump version in package.json to the next patch version.

  We expect the next version to be mostly a patch version. In case you want to release new minor or major version you need to bump it before the release.