polus-railyard

polus-railyard

Template engine for Dockerfiles from Yaml definitions

• Usage
• Commands
• Templating Tutorial

Usage

$ npm install -g polus-railyard
$ railyard COMMAND
running command...
$ railyard (-v|--version|version)
polus-railyard/0.3.2 linux-x64 node-v15.14.0
$ railyard --help [COMMAND]
USAGE
  $ railyard COMMAND
...

Commands

• railyard assemble
• railyard hash
• railyard help [COMMAND]

railyard assemble

Assembles all files required to build a Docker image from Dockerfile template and yaml definitions

USAGE
  $ railyard assemble

OPTIONS
  -a, --additional=additional  Additional stack yaml file
  -b, --base=base              (required) Base stack yaml file
  -p, --path=path              (required) Assembled output folder
  -t, --template=template      (required) Dockerfile template file

DESCRIPTION
  Assembles all files required to build a Docker image from Dockerfile template and yaml definitions

See code: src/commands/assemble.js

railyard hash

Calculates hash from yaml definitions

USAGE
  $ railyard hash

OPTIONS
  -a, --additional=additional  Additional stack yaml files
  -b, --base=base              (required) Base stack yaml file

DESCRIPTION
  Calculates hash from yaml definitions

See code: src/commands/hash.js

railyard help [COMMAND]

display help for railyard

USAGE
  $ railyard help [COMMAND]

ARGUMENTS
  COMMAND  command to show help for

OPTIONS
  --all  see all commands in CLI

See code: @oclif/plugin-help

Templating Tutorial

Getting started with an existing Dockerfile

If you have an existing Dockerfile, i.e.

FROM node:10

...

If you would like to keep track of the Node version in this example, you can substitute it with a variable name placeholder like this:

FROM {{ node_version }}

...

At the same time, you will need to create a yaml file to hold the variable value:

node_version: "node:10"

In the future, you might want to switch to a newer version of node, all you need to do is to update variable in the yaml file, i.e.

node_version: "node:12"

Using lists for tracking dependencies

Besides keeping track of single variables, you are free to use variable arrays in the following format:

list_param:
  - param_1: val_1
  - param_2: val_2
  ...

This could be useful for tracking dependencies in some package manager, where you can have it stored together with a version name, i.e.

RUN pip install {% for package in pip %}{{ package.key }}{% if package.value %}=={{package.value}}{% endif %} {% endfor %}

pip:
  - pandas: 1.0.3
  - numpy: 1.18.3

Here we used multiple nunjucks language constructions:

• for loop: {% for package in pip %} <...> {% endfor %}
• Accessing object attributes: {{ package.key }} and {{package.value}}
• if clause: {% if package.value %} <...> {% endif %} (here we check if version is specified in yaml)

Calculating hash

Before filling in variable into Dockerfile template railyard calculates SHA256-based hash of the yaml definition. name variable is excluded from hash calculation. You may access the hash variable in the template by including {{hash}}.

This feature could be useful when you would like to assign unique image tags for each image based on versions of all included dependencies. One way to achieve that is to include hash label towards the top of the Dockerfile template:

LABEL hash={{ hash }}

This way, when any of the variable used below in the Dockerfile get updated, the whole Dockerfile will be rebuit.

Also, you may call railyard hash in your CI/CD pipeline and use the return value to tag you images upon building/pushing to registry.

Combining multiple yaml files to create combinations of options

railyard offers an option to provide multiple yaml stacks in addition to the base one. You can use that to easily create multiple images based on included features. Let's say you are creating a Dockerfile for packaging machine learning code. You may create base stack yaml which includes Python (base.yaml) and two additional stacks to include Tensorflow (tensorflow.yaml) and Pytorch (pytorch.yaml) respectively.

Then, in the CI/CD pipeline, you may effectively generate all possible combinations of included features, by choosing which stack yamls to include:

railyard assemble -t Dockerfile.template -b base.yaml -p dockerfiles/
railyard assemble -t Dockerfile.template -b base.yaml -a tensorflow.yaml -p dockerfiles/
railyard assemble -t Dockerfile.template -b base.yaml -a pytorch.yaml -p dockerfiles/
railyard assemble -t Dockerfile.template -b base.yaml -a tensorflow.yaml -a pytorch.yaml -p dockerfiles/