umzug

What is umzug?

Umzug is a framework-agnostic migration tool for Node.js. It allows you to manage and execute database migrations, which are essential for evolving your database schema over time in a controlled and consistent manner.

What are umzug's main functionalities?

Running Migrations

This code demonstrates how to set up and run migrations using Umzug with Sequelize as the storage engine. The `up` method runs all pending migrations.

const { Sequelize } = require('sequelize');
const { Umzug, SequelizeStorage } = require('umzug');

const sequelize = new Sequelize('sqlite::memory:');

const umzug = new Umzug({
  migrations: { glob: 'migrations/*.js' },
  context: sequelize.getQueryInterface(),
  storage: new SequelizeStorage({ sequelize }),
  logger: console,
});

(async () => {
  await umzug.up();
})();

Reverting Migrations

This code demonstrates how to revert the last executed migration using the `down` method.

const { Sequelize } = require('sequelize');
const { Umzug, SequelizeStorage } = require('umzug');

const sequelize = new Sequelize('sqlite::memory:');

const umzug = new Umzug({
  migrations: { glob: 'migrations/*.js' },
  context: sequelize.getQueryInterface(),
  storage: new SequelizeStorage({ sequelize }),
  logger: console,
});

(async () => {
  await umzug.down();
})();

Listing Pending Migrations

This code demonstrates how to list all pending migrations using the `pending` method.

const { Sequelize } = require('sequelize');
const { Umzug, SequelizeStorage } = require('umzug');

const sequelize = new Sequelize('sqlite::memory:');

const umzug = new Umzug({
  migrations: { glob: 'migrations/*.js' },
  context: sequelize.getQueryInterface(),
  storage: new SequelizeStorage({ sequelize }),
  logger: console,
});

(async () => {
  const pendingMigrations = await umzug.pending();
  console.log(pendingMigrations);
})();

Other packages similar to umzug

knex

Knex.js is a SQL query builder for Node.js that also includes a built-in migration tool. It is more feature-rich in terms of query building and supports multiple database types, but it is more tightly coupled to its own query builder.

migrate

Migrate is a simple, framework-agnostic migration tool for Node.js. It is less feature-rich compared to Umzug but is very lightweight and easy to use for simple migration needs.

db-migrate

db-migrate is a database migration framework for Node.js that supports multiple databases and includes a CLI tool. It is more opinionated and comes with more built-in features compared to Umzug.

README

Umzug

Umzug is a framework-agnostic migration tool for Node. It provides a clean API for running and rolling back tasks.

Note: master represents the next major version of umzug - v3 - which is currently in beta. For the stable version, please refer to the v2.x branch.

To install the v3-beta package:

npm install umzug@beta

To install the stable package (v2.x):

npm install umzug

Highlights

• Written in TypeScript
  • Built-in typings
  • Auto-completion right in your IDE
  • Documentation right in your IDE
• Programmatic API for migrations
• Built in CLI
• Database agnostic
• Supports logging of migration process
• Supports multiple storages for migration data

Documentation

Minimal Example

The following example uses a Sqlite database through sequelize and persists the migration data in the database itself through the sequelize storage.

// index.js
const { Sequelize } = require('sequelize');
const { Umzug, SequelizeStorage } = require('umzug');

const sequelize = new Sequelize({ dialect: 'sqlite', storage: './db.sqlite' });

const umzug = new Umzug({
  migrations: { glob: 'migrations/*.js' },
  context: sequelize.getQueryInterface(),
  storage: new SequelizeStorage({ sequelize }),
  logger: console,
});

(async () => {
  // Checks migrations and run them if they are not already applied. To keep
  // track of the executed migrations, a table (and sequelize model) called SequelizeMeta
  // will be automatically created (if it doesn't exist already) and parsed.
  await umzug.up();
})();

// migrations/00_initial.js

const { Sequelize } = require('sequelize');

async function up({ context: queryInterface }) {
	await queryInterface.createTable('users', {
		id: {
			type: Sequelize.INTEGER,
			allowNull: false,
			primaryKey: true
		},
		name: {
			type: Sequelize.STRING,
			allowNull: false
		},
		createdAt: {
			type: Sequelize.DATE,
			allowNull: false
		},
		updatedAt: {
			type: Sequelize.DATE,
			allowNull: false
		}
	});
}

async function down({ context: queryInterface }) {
	await queryInterface.dropTable('users');
}

module.exports = { up, down };

Note that we renamed the context argument to queryInterface for clarity. The context is whatever we specified when creating the Umzug instance in index.js.

You can also write your migrations in typescript by using `ts-node` in the entrypoint:

// index.ts
require('ts-node/register')

import { Sequelize } from 'sequelize';
import { Umzug, SequelizeStorage } from 'umzug';

const sequelize = new Sequelize({ dialect: 'sqlite', storage: './db.sqlite' });

const umzug = new Umzug({
  migrations: { glob: 'migrations/*.ts' },
  context: sequelize.getQueryInterface(),
  storage: new SequelizeStorage({ sequelize }),
  logger: console,
});

// export the type helper exposed by umzug, which will have the `context` argument typed correctly
export type Migration = typeof umzug._types.migration;

(async () => {
  await umzug.up();
})();

// migrations/00_initial.ts
import type { Migration } from '..';

// types will now be available for `queryInterface`
export const up: Migration = ({ context: queryInterface }) => queryInterface.createTable(...)
export const down: Migration = ({ context: queryInterface }) => queryInterface.dropTable(...)

See these tests for more examples of Umzug usage, including:

• passing ignore and cwd parameters to the glob instructions
• customising migrations ordering
• finding migrations from multiple different directories
• using non-js file extensions via a custom resolver, e.g. .sql

Usage

Installation

The beta version of Umzug is available on npm by specifying the correct tag:

npm install umzug@beta

Umzug instance

It is possible to configure an Umzug instance by passing an object to the constructor.

const { Umzug } = require('umzug');
const umzug = new Umzug({ /* ... options ... */ });

Detailed documentation for these options are in the UmzugConstructorOptions TypeScript interface, which can be found in src/types.ts.

Getting all pending migrations

You can get a list of pending (i.e. not yet executed) migrations with the pending() method:

const migrations = await umzug.pending();
// returns an array of all pending migrations.

Getting all executed migrations

You can get a list of already executed migrations with the executed() method:

const migrations = await umzug.executed();
// returns an array of all already executed migrations

Executing pending migrations

The up method can be used to execute all pending migrations.

const migrations = await umzug.up();
// returns an array of all executed migrations

It is also possible to pass the name of a migration in order to just run the migrations from the current state to the passed migration name (inclusive).

await umzug.up({ to: '20141101203500-task' });

To limit the number of migrations that are run, step can be used:

await umzug.up({ step: 2 })

Running specific migrations while ignoring the right order, can be done like this:

await umzug.up({ migrations: ['20141101203500-task', '20141101203501-task-2'] });

Reverting executed migration

The down method can be used to revert the last executed migration.

const migration = await umzug.down();
// reverts the last migration and returns it.

To revert more than one migration, you can use step:

await umzug.down({ step: 2 });

It is possible to pass the name of a migration until which (inclusive) the migrations should be reverted. This allows the reverting of multiple migrations at once.

const migrations = await umzug.down({ to: '20141031080000-task' });
// returns an array of all reverted migrations.

To revert all migrations, you can pass 0 as the to parameter:

await umzug.down({ to: 0 });

Reverting specific migrations while ignoring the right order, can be done like this:

await umzug.down({ migrations: ['20141101203500-task', '20141101203501-task-2'] });

Migrations

There are two ways to specify migrations: via files or directly via an array of migrations.

Migration files

A migration file ideally exposes an up and a down async functions. They will perform the task of upgrading or downgrading the database.

module.exports = {
  async up() {
    /* ... */
  },
  async down() {
    /* ... */
  }
};

Migration files can be located anywhere - they will typically be loaded according to a glob pattern provided to the Umzug constructor.

Direct migrations list

You can also specify directly a list of migrations to the Umzug constructor:

const { Umzug } = require('umzug');

const umzug = new Umzug({
  migrations: [
    {
      // the name of the migration is mandatory
      name: '00-first-migration',
      async up({ context }) { /* ... */ },
      async down({ context }) { /* ... */ }
    },
    {
      name: '01-foo-bar-migration',
      async up({ context }) { /* ... */ },
      async down({ context }) { /* ... */ }
    }
  ],
  context: sequelize.getQueryInterface(),
  logger: console,
});

To load migrations in another format, you can use the resolve function:

const { Umzug } = require('umzug')
const { Sequelize } = require('sequelize')
const fs = require('fs')

const umzug = new Umzug({
  migrations: {
    glob: 'migrations/*.up.sql',
    resolve: ({ name, path, context: sequelize }) => ({
      name,
      up: async () => {
        const sql = fs.readFileSync(path).toString()
        return sequelize.query(sql)
      },
      down: async () => {
        // Get the corresponding `.down.sql` file to undo this migration
        const sql = fs.readFileSync(path.replace('.up.sql', '.down.sql')).toString()
        return sequelize.query(sql)
      }
    })
  },
  context: new Sequelize(...),
  logger: console,
});

You can support mixed migration file types, and use umzug's default resolver for javascript/typescript:

const { Umzug } = require('umzug')
const { Sequelize } = require('sequelize')
const fs = require('fs')

const umzug = new Umzug({
  migrations: {
    glob: 'migrations/*.{js,ts,up.sql}',
    resolve: (params) => {
      if (!params.path.endsWith('.sql')) {
        return Umzug.defaultResolver(params)
      }
      const { context: sequelize } = params
      return {
        name: params.name,
        up: async () => {
          const sql = fs.readFileSync(params.path).toString()
          return sequelize.query(sql)
        },
        down: async () => {
          // Get the corresponding `.down.sql` file to undo this migration
          const sql = fs.readFileSync(params.path.replace('.up.sql', '.down.sql')).toString()
          return sequelize.query(sql)
        }
      }
    },
  },
  logger: console,
  context: new Sequelize(...),
});

The glob syntax allows loading migrations from multiple locations:

const { Umzug } = require('umzug')
const { Sequelize } = require('sequelize')

const umzug = new Umzug({
  migrations: {
    glob: '{first-folder/*.js,second-folder-with-different-naming-convention/*.js}',
  },
  context: new Sequelize(...),
  logger: console,
});

Note on migration file sorting:

• file matches, found using glob, will be lexicographically sorted based on their paths
  • so if your migrations are one/m1.js, two/m2.js, three/m3.js, the resultant order will be one/m1.js, three/m3.js, two/m2.js
  • similarly, if your migrations are called m1.js, m2.js, ... m10.js, m11.js, the resultant ordering will be m1.js, m10.js, m11.js, ... m2.js
• The easiest way to deal with this is to ensure your migrations appear in a single folder, and their paths match lexicographically with the order they should run in
• If this isn't possible, the ordering can be customised using .extend(...) (see below)

Upgrading from v2.x

The Umzug class should be imported as a named import, i.e. import { Umzug } from 'umzug'.

The MigrationMeta type, which is returned by umzug.executed() and umzug.pending(), no longer has a file property - it has a name and optional path - since migrations are not necessarily bound to files on the file system.

The migrations.glob parameter replaces path, pattern and traverseDirectories. It can be used, in combination with cwd and ignore to do much more flexible file lookups. See https://npmjs.com/package/glob for more information on the syntax.

The migrations.resolve parameter replaces customResolver. Explicit support for wrap and nameFormatter has been removed - these can be easily implemented in a resolve function.

The constructor option logging is replaced by logger to allow for warn and error messages in future. NodeJS's global console object can be passed to this. To disable logging, replace logging: false with logger: undefined.

The Umzug#execute method is removed. Use Umzug#up or Umzug#down.

The options for Umguz#up and Umzug#down have changed:

• umzug.up({ to: 'some-name' }) and umzug.down({ to: 'some-name' }) are still valid.
• umzug.up({ from: '...' }) and umzug.down({ from: '...' }) are no longer supported. To run migrations out-of-order (which is not generally recommended), you can explicitly use umzug.up({ migrations: ['...'] }) and umzug.down({ migrations: ['...'] }).
• name matches must be exact. umzug.up({ to: 'some-n' }) will no longer match a migration called some-name.
• umzug.down({ to: 0 }) is still valid but umzug.up({ to: 0 }) is not.
• umzug.up({ migrations: ['m1', 'm2'] }) is still valid but the shorthand umzug.up(['m1', 'm2']) has been removed.
• umzug.down({ migrations: ['m1', 'm2'] }) is still valid but the shorthand umzug.down(['m1', 'm2']) has been removed.
• umzug.up({ migrations: ['m1', 'already-run'] }) will throw an error, if already-run is not found in the list of pending migrations.
• umzug.down({ migrations: ['m1', 'has-not-been-run'] }) will throw an error, if has-not-been-run is not found in the list of executed migrations.
• umzug.up({ migrations: ['m1', 'm2'], rerun: 'ALLOW' }) will re-apply migrations m1 and m2 even if they've already been run.
• umzug.up({ migrations: ['m1', 'm2'], rerun: 'SKIP' }) will skip migrations m1 and m2 if they've already been run.
• umzug.down({ migrations: ['m1', 'm2'], rerun: 'ALLOW' }) will "revert" migrations m1 and m2 even if they've never been run.
• umzug.down({ migrations: ['m1', 'm2'], rerun: 'SKIP' }) will skip reverting migrations m1 and m2 if they haven't been run or are already reverted.
• umzug.up({ migrations: ['m1', 'does-not-exist', 'm2'] }) will throw an error if the migration name is not found. Note that the error will be thrown and no migrations run unless all migration names are found - whether or not rerun: 'ALLOW' is added.

The context parameter replaces params, and is passed in as a property to migration functions as an options object, alongs side name and path. This means the signature for migrations, which in v2 was (context) => Promise<void>, has changed slightly in v3, to ({ name, path, context }) => Promise<void>. The resolve function can also be used to upgrade your umzug version to v3 when you have existing v2-compatible migrations:

const { Umzug } = require('umzug');

const umzug = new Umzug({
  migrations: {
    glob: 'migrations/umzug-v2-format/*.js',
    resolve: ({name, path, context}) => {
      // Adjust the migration from the new signature to the v2 signature, making easier to upgrade to v3
      const migration = require(path)
      return { name, up: async () => migration.up(context), down: async () => migration.down(context) }
    }
  },
  context: sequelize.getQueryInterface(),
  logger: console,
});

Similarly, you no longer need migrationSorting, you can use Umzug#extend to manipulate migration lists directly:

const { Umzug } = require('umzug');

const umzug =
  new Umzug({
    migrations: { glob: 'migrations/**/*.js' },
    context: sequelize.getQueryInterface(),
  })
  .extend(migrations => migrations.sort((a, b) => b.path.localeCompare(a.path)));

Storages

Storages define where the migration data is stored.

JSON Storage

Using JSONStorage will create a JSON file which will contain an array with all the executed migrations. You can specify the path to the file. The default for that is umzug.json in the working directory of the process.

Detailed documentation for the options it can take are in the JSONStorageConstructorOptions TypeScript interface, which can be found in src/storage/json.ts.

Memory Storage

Using memoryStorage will store migrations with an in-memory array. This can be useful for proof-of-concepts or tests, since it doesn't interact with databases or filesystems.

It doesn't take any options, just import the memoryStorage function and call it to return a storage instance:

import { Umzug, memoryStorage } from 'umzug'

const umzug = new Umzug({
  migrations: ...,
  storage: memoryStorage(),
  logger: console,
})

Sequelize Storage

Using SequelizeStorage will create a table in your SQL database called SequelizeMeta containing an entry for each executed migration. You will have to pass a configured instance of Sequelize or an existing Sequelize model. Optionally you can specify the model name, table name, or column name. All major Sequelize versions are supported.

Detailed documentation for the options it can take are in the _SequelizeStorageConstructorOptions TypeScript interface, which can be found in src/storage/sequelize.ts.

MongoDB Storage

Using MongoDBStorage will create a collection in your MongoDB database called migrations containing an entry for each executed migration. You will have either to pass a MongoDB Driver Collection as collection property. Alternatively you can pass a established MongoDB Driver connection and a collection name.

Detailed documentation for the options it can take are in the MongoDBStorageConstructorOptions TypeScript interface, which can be found in src/storage/mongodb.ts.

Custom

In order to use a custom storage, you can pass your storage instance to Umzug constructor.

class CustomStorage {
  constructor(...) {...}
  logMigration(...) {...}
  unlogMigration(...) {...}
  executed(...) {...}
}

const umzug = new Umzug({ storage: new CustomStorage(...), logger: console })

Your instance must adhere to the UmzugStorage interface. If you're using TypeScript you can ensure this at compile time, and get IDE type hints by importing it:

import { UmzugStorage } from 'umzug'

class CustomStorage implements UmzugStorage {
  /* ... */
}

Creating migrations - API

Umzug includes an optional helper for generating migration files. This will, by default, include a timestamp-based prefix and generate a file in the same folder as the most recent existing migration. It also includes some safety checks to make sure migrations aren't created with ambiguous ordering, and that they will be picked up by umzug when applying migrations. It's often most convenient to create files using the CLI helper, but the equivalent API also exists on an umzug instance:

await umzug.create({ name: 'my-new-migration.js' })

Umzug ships with a barebones default template for js, ts and sql migration files, but you can specify the template for your project when constructing an umzug instance. The template property should be a function which receives a filepath string, and returns an array of [filepath, content] pairs. Usually, just one pair is needed, but a second could be used to include a "down" migration in a separate file:

const umzug = new Umzug({
  migrations: ...,
  template: filepath => [
    [
      filepath,
      dedent`
        import { Migration } from '@my-team/migration-type'
        import { logger } from '@my-team/custom-logger'

        export const up: Migration = params => {
          logger.info('running migration', params.name)
        }

        export const down: Migration = params => {
          logger.info('reverting migration', params.name)
        }
      `
    ],
  ]
})

To create blank migration files, return an empty array:

const umzug = new Umzug({
  migrations: ...,
  template: () => [],
})

Events

Umzug is an EventEmitter. Each of the following events will be called with name and migration as arguments. Events are a convenient place to implement application-specific logic that must run around each migration:

• migrating - A migration is about to be executed.
• migrated - A migration has successfully been executed.
• reverting - A migration is about to be reverted.
• reverted - A migration has successfully been reverted.

CLI

Umzug has an optional built-in CLI helper. To use it, call the .runAsCLI() method from an Umzug instance:

// my-umzug-migrator.js
const { Umzug } = require('umzug')

exports.umzug = new Umzug({ ... })

if (require.main === module) {
  umzug.runAsCLI()
}

This script is now a runnable CLI program, complete with help text. You can run node my-umzug-migrator.js --help to see how to use it. It'll print something like

usage: <script> [-h] <command> ...

Umzug migrator

Positional arguments:
  <command>
    up        Applies pending migrations
    down      Revert migrations
    pending   Lists pending migrations
    executed  Lists executed migrations
    create    Create a migration file

Optional arguments:
  -h, --help  Show this help message and exit.

For detailed help about a specific command, use: <script> <command> -h

Running migrations

node my-umzug-migrator up and node my-umzug-migrator down apply and revert migrations respectively. They're the equivalent of the .up() and .down() methods.

Use node my-umzug-migrator up --help and node my-umzug-migrator down --help for options (running "to" a specific migration, passing migration names to be run explicitly, and specifying the rerun behavior):

Up:

usage: <script> up [-h] [--to NAME] [--step COUNT] [--migration NAME]
                   [--rerun {THROW,SKIP,ALLOW}]
                   

Performs all migrations. See --help for more options

Optional arguments:
  -h, --help            Show this help message and exit.
  --to NAME             All migrations up to and including this one should be 
                        applied.
  --step COUNT          Run this many migrations. If not specified, all will 
                        be applied.
  --migration NAME      List of migrations to be applied
  --rerun {THROW,SKIP,ALLOW}
                        Specify what action should be taken when a migration 
                        that has already been applied is passed. The default 
                        value is "THROW".

Down:

usage: <script> down [-h] [--to NAME] [--step COUNT] [--migration NAME]
                     [--rerun {THROW,SKIP,ALLOW}]
                     

Undoes previously-applied migrations. By default, undoes the most recent 
migration only. Use --help for more options. Useful in development to start 
from a clean slate. Use with care in production!

Optional arguments:
  -h, --help            Show this help message and exit.
  --to NAME             All migrations up to and including this one should be 
                        reverted. Pass "0" to revert all.
  --step COUNT          Run this many migrations. If not specified, one will 
                        be reverted.
  --migration NAME      List of migrations to be reverted
  --rerun {THROW,SKIP,ALLOW}
                        Specify what action should be taken when a migration 
                        that has already been reverted is passed. The default 
                        value is "THROW".

Listing migrations

node my-umzug-migrator pending # list migrations yet to be run
node my-umzug-migrator executed # list migrations that have already run

node my-umzug-migrator pending --json # list pending migrations including names and paths, in a json array format
node my-umzug-migrator executed --json # list executed migrations including names and paths, in a json array format

node my-umzug-migrator pending --help # show help/options
node my-umzug-migrator executed --help # show help/options

usage: <script> pending [-h] [--json]

Prints migrations returned by `umzug.pending()`. By default, prints migration 
names one per line.

Optional arguments:
  -h, --help  Show this help message and exit.
  --json      Print pending migrations in a json format including names and 
              paths. This allows piping output to tools like jq. Without this 
              flag, the migration names will be printed one per line.

usage: <script> executed [-h] [--json]

Prints migrations returned by `umzug.executed()`. By default, prints 
migration names one per line.

Optional arguments:
  -h, --help  Show this help message and exit.
  --json      Print executed migrations in a json format including names and 
              paths. This allows piping output to tools like jq. Without this 
              flag, the migration names will be printed one per line.

Creating migrations - CLI

Usually, migrations correspond to files on the filesystem. The CLI exposes a way to create migration files easily:

node my-umzug-migrator create --name my-migration.js

This will create a file with a name like 2000.12.25T12.34.56.my-migration.js in the same directory as the most recent migration file. If it's the very first migration file, you need to specify the folder explicitly:

node my-umzug-migrator create --name my-migration.js --folder path/to/directory

The timestamp prefix can be customized to be date-only or omitted, but be aware that it's strongly recommended to ensure your migrations are lexicographically sortable so it's easy for humans and tools to determine what order they should run in - so the default prefix is recommended.

Use node my-umzug-migrator create --help for more options:

usage: <script> create [-h] --name NAME [--prefix {TIMESTAMP,DATE,NONE}]
                       [--folder PATH] [--allow-extension EXTENSION]
                       [--skip-verify] [--allow-confusing-ordering]
                       

Generates a placeholder migration file using a timestamp as a prefix. By 
default, mimics the last existing migration, or guesses where to generate the 
file if no migration exists yet.

Optional arguments:
  -h, --help            Show this help message and exit.
  --name NAME           The name of the migration file. e.g. my-migration.js, 
                        my-migration.ts or my-migration.sql. Note - a prefix 
                        will be added to this name, usually based on a 
                        timestamp. See --prefix
  --prefix {TIMESTAMP,DATE,NONE}
                        The prefix format for generated files. TIMESTAMP uses 
                        a second-resolution timestamp, DATE uses a 
                        day-resolution timestamp, and NONE removes the prefix 
                        completely. The default value is "TIMESTAMP".
  --folder PATH         Path on the filesystem where the file should be 
                        created. The new migration will be created as a 
                        sibling of the last existing one if this is omitted.
  --allow-extension EXTENSION
                        Allowable extension for created files. By default .js,
                         .ts and .sql files can be created. To create txt 
                        file migrations, for example, you could use '--name 
                        my-migration.txt --allow-extension .txt' This 
                        parameter may alternatively be specified via the 
                        UMZUG_ALLOW_EXTENSION environment variable.
  --skip-verify         By default, the generated file will be checked after 
                        creation to make sure it is detected as a pending 
                        migration. This catches problems like creation in the 
                        wrong folder, or invalid naming conventions. This 
                        flag bypasses that verification step.
  --allow-confusing-ordering
                        By default, an error will be thrown if you try to 
                        create a migration that will run before a migration 
                        that already exists. This catches errors which can 
                        cause problems if you change file naming conventions. 
                        If you use a custom ordering system, you can disable 
                        this behavior, but it's strongly recommended that you 
                        don't! If you're unsure, just ignore this option.

License

See the LICENSE file