nx-remotecache-custom

The future of nx-remotecache-*

Nrwl Nx just introduced us to Powerpack. It's the clear successor of the remote cache setup and officially supports custom caching solutions. This will mark the end of the nx-remotecache-* packages as custom caching solutions based on the filesystem won't work anymore starting with Nx 21.

Powerpack fills exactly the void that nx-remotecache-custom filled. Therefore I'm happy to give this topic back to the creators behind Nx. Thanks for the great ride – we reached over 114,000 weekly downloads on npm as I'm writing this 🥳

Feel free to read the Introduction to Powerpack by Jeff yourself. If you have any further questions checkout the pinned GitHub issue.

Cheers, Niklas 👋

nx-remotecache-custom

nx-remotecache-custom is a simple package which includes a helper function to create custom nx remote cache implementations. You'll only need to add functions for:

1. storing a file / buffer
2. retrieving a file / buffer
3. checking if a file / buffer exists

and createCustomRunner() is taking care of everything else. Not convinced yet? The package will also:

• Print beautiful & colored nx-style messages to the console 💅🎆
• Allow you to execute asynchronous code in the setup phase of your runner 🤖
• Handle all thrown errors ➡ No broken builds to offline remote caches 🚀
• Automagically zip all the cached files ➡ Minimal storage & traffic consumption 📦
• Provide a small defined and documented API 📚

Compatability

Nx	Remote Cache
>= 21	Deprecated
>= 20.0.0 < 21	>= 20.0.0
>= 19.0.0 < 20	>= 19.0.0 < 20
>= 18.0.0 < 19	>= 18.0.0 < 19
>= 17.0.0 < 18	>= 17.0.0 < 18
>= 16.9.0 < 17	>= 5.0.0 < 17
< 16.9.0	< 5.0.0

Usage

npm i nx-remotecache-custom

// define custom parameters for your nx.json here
interface MyRunnerOptions {
  remoteUrl: string;
}

export default createCustomRunner<MyRunnerOptions>(async (options) => {
  // initialize environment variables from dotfile
  initEnv(options);

  // initialize the connection to your remote storage here
  const myStorage = new MyStorage(options.remoteUrl);

  return {
    // name is used for logging purposes
    name: "My Storage",

    // fileExists checks whether a file exists on your remote storage
    fileExists: (filename) => myStorage.exists(filename),

    // retrieveFile downloads a file from your remote storage
    retrieveFile: (filename) => myStorage.download(filename),

    // storeFile uploads a file from a buffer to your remote storage
    storeFile: (filename, buffer) => myStorage.upload(filename, buffer),
  };
});

{
  "name": "nx-remotecache-mystorage",
  "main": "index.js"
}

After this your package is already ready for usage. Publish it to npm (or an internal registry) and consume it in your client library. Install it and adjust your nx.json to use the newly created runner:

"tasksRunnerOptions": {
  "default": {
    "runner": "nx-remotecache-mystorage",
    "options": {
      "remoteUrl": "http://127.0.0.1:1337",
      "cacheableOperations": ["build", "test", "lint", "e2e"]
    }
  }
}

For a more in-depth code example you can take a look at the implementation of nx-remotecache-azure which uses this package to implement a nx cache on the Azure Blob Storage.

Advanced Configuration

Option	Environment Variable / .env	Description
name	NXCACHE_NAME	Set to provide task runner name for logging. Overrides name provided in implementation.
verbose		Set to receive full stack traces whenever errors occur. Best used for debugging. Default: false
silent		Set to mute success and info logs. Default: false
read	NXCACHE_READ	Set to enable / disable reading from the remote cache. Default: true
write	NXCACHE_WRITE	Set to enable / disable writing to the remote cache. Default: true
dotenv		Set to false to disable reading .env into process.env. Default: true
dotenvPath		Set to read .env files from a different folder.

"tasksRunnerOptions": {
  "default": {
    "options": {
      "name": "My Storage",
      "verbose": true,
      "silent": true
    }
  }
}

All Custom Runners

Runner	Storage
nx-remotecache-azure	Azure Blob Storage
@pellegrims/nx-remotecache-s3	S3 Storage
nx-remotecache-minio	MinIO Storage
@vercel/remote-nx	Vercel Cache
nx-remotecache-redis	Redis Cache

... and many more!