@springernature/assets-pipeline

Assets pipeline

Springer Nature assets pipeline library is a tool to help you design frontend assets build pipelines for your applications.
It is rather opinionated toward tools and configurations we favour here at Springer Nature (e.g. sass, esbuild, as well as the eslint, stylelint linters).

A few benefits to adopt this library:

• Play by the Springer Nature recommended guide lines
• Consolidate the frontend assets pipeline authoring experience across our applications
• Maintenance and evolution of the library is shared across our discipline. This effort is indeed centralised at the library level, rather than being duplicated on every projects.

Requirements

This package requires:

• Node version 20 or greater.
  Please have a look at our open source support page for details on which versions of node we support, and why.
• NPM 5+
• Its peer dependencies to be installed in the host project/machine.

Installation

This library aims to be installed as a development dependency, and can be install via npm:

npm install -D @springernature/assets-pipeline

Usage

Once the module installed, you then have the sn-assets-pipeline command at your disposal.

With a local installation, the sn-assets-pipeline command will not be available in your system path or you can't use it directly from the command line. Instead, the local installation of sn-assets-pipeline can be run by calling it from within an npm script (such as npm run build or npm run serve) or using npx sn-assets-pipeline.

It is meant to be used like:

sn-assets-pipeline <command>

where <command> is one of the following:

• help: Displays the command's usage
• build: Runs the pipeline
• watch: Runs the pipeline and re-run upon changes

The command is driven by a configuration: the assets pipeline configuration.

There are 3 ways to provide this configuration and they follow the below priority:

1. In a seperate file from which the path is referenced in a ASSETS_PIPELINE_CONFIG environment variable. The path is resolved relatively from the host project's directory containing the package.json.
2. In a seperate assets-pipeline.config.json file in the host project's
3. As an assetsPipeline property of the host project's package.json file directory

Each projects has it's own structure and therefore there is no such thing as a default configuration. Please refer to the Configuration section to setup you very own assets pipeline configuration.

Tasks supported by the assets pipeline library

clean task

Taken an array of file/directory paths as a targets property, it deletes the listed files from the file system and empty the listed directories.

Using the clean task reveals useful when aiming to keep your distribution directory current, reflecting the latest state of your assets, and free from obsolete assets that may lead to potential errors.

Example of a clean task configuration

copyAssets task

Taken an array of file/directories assets described by their source and destination properties, it makes a copy of each asset from its source to its destination. If the parent directory, and/or any of its ancestors, of the destination does not exist, they will be created.

Example of a copyAssets task configuration

lintSass task

Taken an array of glob file patterns, it runs stylelint over them to lint any matching Sass files they match.

Linting relies on host project's .stylelintrc configuration file.

Example of a lintSass task configuration

buildCss task

Taken a source, destination among other Sass compilation options, it compiles Sass files into CSS files.

Example of a buildCss task configuration

lintJs task

Taken an array of glob file patterns, it runs eslint over them to lint any matching JavaScript files they match.

Linting relies on host project's .eslintrc configuration file.

Example of a lintJs task configuration

buildJs task

Taken an array of bundles, it compiles these into browser friendly JavaScript using esbuild. Additionally it amends an assets manifest file to include the compiled javascript files.

Note: In case you want to exclude a bundle from the fingerprinting process and therefore from being stored into the assets manifest file , you can create a dedicated buildJs task and ommit the assetsManifestFile.

Example of a buildJs task configuration

watchFiles task

Taken an array of file/directory patterns to watch (i.e watchPatterns), it does not do anything else than triggering the browser Live Reload.

That reveals useful for reloading after a change happened to "non-assets" resources (e.g. view context-data, handlebars template...).

Note: Depending on how broad the spectrum of your watchPatterns is, this may be cluttering your output. That is a perfect use case for the quietWatch option.

Example of a watchFiles task configuration

Assets pipeline configuration

Core properties

tasks property

An array of tasks objets, it represents the orchestration of tasks for your pipeline. Therefore the order of tasks in the array dictates the order of execution.
Keep this in mind especially if you have a task that depends on the outcome of another.
For instance and a task of type buildJs may depend on the outcome of a copyAssets task if the copy operation creates files that needs then to be transpiled in the buildJs task.

Tasks types should be picked among tasks supported by the assets pipeline library.

A full assets pipeline configuration example

watchOptions properties

watchOptions.buildFirst property

By default, the watch command will start by running a build and only then sets itself up to watch for changes. In case you want to decouple the build operation from the watch command, you can configure the watchOptions.buildFirst boolean property as in the below example:

"watchOptions": {
  "buildFirst": false
}

watchOptions.ignorePatterns property

An array of glob file/directory patterns. Any change from files and directories matching these patterns will be ignored by the watch command, i.e will not trigger any task and browser live reload.

Below is an example that instructs to not trigger a Live Reload in case of changes made to testing files:

"watchOptions": {
  "ignorePatterns": [
  "*.test.js",
  "*.spec.js"
  ]
}

watchOptions.liveReload property

If you pass a browser-sync configuration here then live reload will be implemented as part of the watch command.

The port property corresponds to the port the browser-sync server should be run from.

Below is an example of a suggested live reload configuration:

"watchOptions": {
  "liveReload": {
      "online": false,
      "open": false,
      "port": 3001,
      "ui": false,
      "proxy": "localhost:8080",
      "logPrefix": "Live reload"
    }
}

Tasks anatomy

The common properties for a tasks are:

• type (required): String to be picked among supported tasks:
  • clean
  • copyAssets
  • lintSass
  • buildCss
  • lintJs
  • buildJs
  • watchFiles
• name (optional): an optional name for the task, which will be displayed in the console output of your Terminal
• watchPatterns (optional): Array of file/directory patterns to match assets that needs to be monitored for changes in order to re-trigger the execution of the task and a browser Live Reload. This option is consumed by the watch command.
• quietWatch (optional): Option consumed by the watch command. If true, it will not output messages like "Executing task X due to Asset changes in PATTERN".

In the below examples we will be focusing on the specific properties.

Let's go over the different supported tasks and learn about them by example.

clean task example

Below is an example of clean task configuration that instructs a clean up of the dist directory, as well as the deletion of the foo.bar, through its targets property.

{
  "name": "Clean dist directory and foo.bar file",
  "type": "clean",
  "targets": [
    "dist",
    "foo.bar"
  ]
},

copyAssets task example

Below is an example of copyAssets task configuration that instructs the copy of a bunch of assets toward a dist directory.

Each asset in the assets array is described by a source and destination property that are hopefully self-explanatory.

The watchPatterns option is taken into account in case of running the watch command.
Below it instructs for a new execution of the copyAssets task in case any change happens under src/assets/img/logos, src/assets/img/icons, src/assets/fonts or node_modules/@springernature/elements directory.

{
  "name": "Copy necessary assets",
  "type": "copyAssets",
  "assets": [
    {
      "source": "src/assets/img/logos",
      "destination": "dist/img/logos"
    },
    {
      "source": "src/assets/img/icons",
      "destination": "dist/img/icons"
    },
    {
      "source": "src/assets/fonts",
      "destination": "dist/fonts"
    },
    {
      "source": "node_modules/@springernature/elements/themes/springernature/img/icons/eds-i-user-single-medium.svg",
      "destination": "dist/img/icons/eds-i-user-single-medium.svg"
    }
  ],
  "watchPatterns": [
    "src/assets/img/{logos,icons}/**/*.*",
    "src/assets/fonts/**/*.*",
    "node_modules/@springernature/elements/**/*.*"
  ]
}

lintSass task example

Below is an example of lintSass task configuration that instructs, through the watchPatterns property, the linting of a bunch of Sass file from 2 different directories: src/css and src/packages.

The watchPatterns has a double function for that task property as it also serves as a reference for the watch command to know what to monitor for changes.

The exitOnError property lets you decide if the pipeline should break upon lint issue ("exitOnError": true), or if it should continue, printing out silentely the lint issues ("exitOnError": false). It defaults to true.

{
  "name": "Lint sass files with stylelint",
  "type": "lintSass",
  "watchPatterns": [
    "src/css/**/*.scss",
    "src/packages/**/*.scss"
  ],
  "exitOnError": true
}

buildCss task example

Below is an example of buildCss task configuration that instructs the processing of a bunch of Sass files, hosted into source directory src/css, toward a dist directory.

The sassOptions is a valid Sass options configuration as by Dart Sass documentation. The one from the example instructs the creation of source map files for the generated CSS files.

The autoprefixer set to true, instructs to autoprefix CSS properties according to existing .browserslistrc file or browserlist property in package.json.

The assetsManifestFile instructs to create or update the specified assets manifest file with the generated assets.

The watchPatterns option is taken into account in case of running the watch command.
Below it instructs for a new execution of the buildCss task in case any change happens under src/css or src/packages directory.

{
  "name": "Compile sass files with dart Sass",
  "type": "buildCss",
  "source": "src/css",
  "destination": "dist",
  "sassOptions": {
    "sourceMap": true
  },
  "autoprefixer": true,
  "assetsManifestFile": "dist/assets.json",
  "watchPatterns": [
    "src/css/**/*",
    "src/packages/**/*"
  ]
}

lintJs task example

Below is an example of lintJs task configuration that instructs, through the watchPatterns property, the linting of a bunch of JavaScript file from 2 different directories: src/js and src/shared.

The watchPatterns has a double function for that task property as it also serves as a reference for the watch command to know what to monitor for changes.

The exitOnError property lets you decide if the pipeline should break upon lint issue ("exitOnError": true), or if it should continue, printing out silentely the lint issues ("exitOnError": false). It defaults to true.

{
  "name": "Lint JS files with eslint",
  "type": "lintJs",
  "watchPatterns": [
    "src/js/**/*",
    "src/shared/**/*"
  ],
  "exitOnError": true
},

buildJs task example

Below is an example of buildJs task configuration that instructs the creating of 2 JavaScript bundles: dist/main.js and dist/publication.js meant to be served by the browser.

The bundles property is an array of ESBuild configurations.

Important: So far we only support the outfile approach to ESBuild configuration which is only applicable if there is a single entry point in the entryPoints array. We may broaden the types of configuration we support in future iterations if the need arises.

The buildJs task, is using a bunch of sensible defaults that hopefully everyone can agree on:

//...

// Deduced from the frontend playbook
target: [
	'firefox67',
	'chrome76',
	'safari12',
	'edge79',
	'opera62'
],
format: 'iife',
sourcemap: true,
bundle: true,
// Minifying does not make sense in development environment
minify: process.env.NODE_ENV !== 'development'

// ...

These defaults can be overriden in each of your bundles entries, but at your own risk.

The assetsManifestFile instructs to create or update the specified assets manifest file with the generated bundles.

The watchPatterns option is taken into account in case of running the watch command.
Below it instructs for a new execution of the buildJs task in case any change happens under src/js or src/shared directory.

{
  "name": "Transpile and bundle JavaScript files with esbuild",
  "type": "buildJs",
  "bundles": [
    {
      "entryPoints": [
        "src/js/main.js"
      ],
      "outfile": "dist/main.js"
    },
    {
      "entryPoints": [
        "src/js/publication.js"
      ],
      "outfile": "dist/publication.js"
    }
  ],
  "assetsManifestFile": "dist/assets.json",
  "watchPatterns": [
    "src/js/**/*",
    "src/shared/**/*"
  ]
}

watchFiles task example

Below is an example of watchFiles task configuration that instructs a browser Live Reload upon changes under the handlebars or the context-data directories.

{
  "type": "watchFiles",
  "name": "Watch for changes in templates and/or context data",
  "quietWatch": true,
  "watchPatterns": [
    "handlebars/**/*.*",
    "context-data/**/*.*"
  ]
},

A full assets pipeline configuration example

{
  "tasks": [
    {
      "name": "Clean dist directory",
      "type": "clean",
      "targets": [
        "dist"
      ]
    },
    {
      "name": "Copy necessary assets",
      "type": "copyAssets",
      "assets": [
        {
          "source": "src/assets/img/logos",
          "destination": "dist/img/logos"
        },
        {
          "source": "src/assets/img/favicons",
          "destination": "dist/img/favicons"
        },
        {
          "source": "src/assets/fonts",
          "destination": "dist/fonts"
        },
        {
          "source": "src/assets/img/icons",
          "destination": "dist/img/icons"
        },
        {
          "source": "node_modules/@springernature/elements/themes/springernature/img/icons/eds-i-user-single-medium.svg",
          "destination": "dist/img/icons/eds-i-user-single-medium.svg"
        },
        {
          "source": "node_modules/@springernature/elements/themes/springernature/img/icons/eds-i-chevron-right-small.svg",
          "destination": "dist/img/icons/eds-i-chevron-right-small.svg"
        }
      ],
      "watchPatterns": [
        "src/assets/img/{logos,favicons,icons,illustrations}/**/*.*",
        "src/assets/fonts/**/*.*"
      ]
    },
    {
      "name": "Lint sass files with stylelint",
      "type": "lintSass",
      "watchPatterns": [
        "src/css/**/*.scss",
        "src/packages/**/*.scss"
      ],
      "exitOnError": true
    },
    {
      "name": "Compile sass files with dart Sass",
      "type": "buildCss",
      "source": "src/css",
      "watchPatterns": [
        "src/css/**/*",
        "src/packages/**/*"
      ],
      "destination": "dist",
      "sassOptions": {
        "sourceMap": true
      },
      "autoprefixer": true,
      "assetsManifestFile": "dist/assets.json"
    },
    {
      "name": "Lint JS files with eslint",
      "type": "lintJs",
      "watchPatterns": [
        "src/js/**/*",
        "src/shared/**/*",
        "tooling/**/*"
      ],
      "exitOnError": false
    },
    {
      "name": "Transpile and bundle Javascript files with esbuild",
      "type": "buildJs",
      "watchPatterns": [
        "src/js/**/*"
      ],
      "bundles": [
        {
          "entryPoints": [
            "src/js/main.js"
          ],
          "outfile": "dist/main.js"
        },
        {
          "entryPoints": [
            "src/js/publication-details.js"
          ],
          "outfile": "dist/publication-details.js"
        }
      ],
      "assetsManifestFile": "dist/assets.json"
    },
    {
      "name": "Watch for changes in the express server",
      "type": "watchFiles",
      "quietWatch": true,
      "watchPatterns": [
        "src/server/**/*",
        "src/view/**/*",
        "src/packages/**/*",
        "tooling/**/*",
        "fixtures/**/*",
        "context-data/**/*"
      ]
    }
  ],
  "watchOptions": {
    "buildFirst": false,
    "ignorePatterns": [
      ".git",
      "node_modules/**/node_modules",
      "*.test.js",
      "*.spec.js"
    ],
    "liveReload": {
      "online": false,
      "open": false,
      "port": 8081,
      "ui": false,
      "proxy": "local-app.springernature.com:8080",
      "logPrefix": "Live reload"
    }
  }
}

What is the assets manifest file?

A recommended pratice is to fingerprint your assets so that you can cache these files indefinitely. This reduces the number of request the browser needs to make.
Fingerprinting is the operation to suffix file names with a hash of the content of the file (e.g. main-core-f7f9f66.css is the fingerprinted file name for main-core.css).

In order to avoid the need for developers to patch any reference of fingerprinted assets in the code, the idea of an assets manifest file arose. It contains a JSON object from which fingerprinted filenames can easily be resolved by your application from their original name.

This manifest file can be created/amended while building the project's assets. The current assets pipeline library offers, as part of its buildCss and buildJs tasks an assetsManifestFile property that lets you provide the path for this file.

The below example file has been generated at build time while building the following assets: main-core.css, main-enhanced.css, main.js and publication.js.

{
  "main-core": {
    "css": "main-core-f7f9f66.css"
  },
  "main-enhanced": {
    "css": "main-enhanced-1349dc0.css"
  },
  "main": {
    "js": "main-5701042.js"
  },
  "publication": {
    "js": "publication-fbf5769.js"
  }
}

Taken you would store the content of this manifest file in an assets variable made available in a handlebars template; you could reference main-core.css this way:

<link rel="stylesheet" href="/{{assets.main-core.css}}">

This would then render in the page source HTML as:

<link rel="stylesheet" href="/main-core-f7f9f66.css">