dbsync - npm Package Compare versions

@@ -6,4 +6,4 @@ ## dbsync

At its most basic level, dbsync is a command line tool that scans a path for files to run as SQL migrations. Migrations

found which have not been (successfully) run before will be run in ascending order based on path/filename. Various

At its most basic level, dbsync is a command line tool that scans a path for files to run as SQL migrations. Migrations

found which have not been (successfully) run before will be run in ascending order based on path/filename. Various

options can be used to change those basic behaviors.

@@ -23,14 +23,13 @@

Any time a change in the code requires a change in the db structure, a transformation of the data, and/or a small

amount of new data to be inserted, it should happen as a SQL "migration".

amount of new data to be inserted, it should happen as a SQL "migration". The following bullets describe migrations

as they are generally used, and the default behaviors for dbsync; however, all of those behaviors can be altered via

the command line options described later.

Generally, once a migration has been applied to a given database, it will not be applied again to that same database.

Migration files are not inspected for changes; if the filename is the same as a previously successful migration, it

* Generally, once a migration has been applied to a given database, it will not be applied again to that same database.

* Migration files are not inspected for changes; if the filename is the same as a previously successful migration, it

will not be run again.

A migration file will either be applied in its entirety or not at all. If a migration is not applied (due to an

* A migration file will either be applied in its entirety or not at all. If a migration is not applied (due to an

error), no more migrations will be attempted, and the failed migration will be attempted again on next execution.

* Migrations are ordered; they are applied in alphabetical order by filename, including path.

Migrations are ordered; they are applied in alphabetical order by filename, including path.

#### Using dbsync to perform migrations

@@ -57,65 +56,85 @@

```

--help, -h Show help

--path, -p Directory to scan for migration files; this part of the file path will not

be used when determining whether a migration has been run before.

[required]

--client, --db A db client string, as appropriate to pass to knex in the initialization

object: http://knexjs.org/#Installation-client [required]

--connection, --conn Additional db connection options, as appropriate to pass to knex in the

initialization object: http://knexjs.org/#Installation-client In order to

set subproperties, use dot notation as in these examples:

--client=pg --connection=postgres://user:pw@host:port/dbname?options

--client=mysql --connection.user=user --connection.host=localhost

--client=sqlite3 --connection.filename=./mydb.sqlite [required]

--table, -t Table name for tracking migrations [default: "dbsync_migrations"]

--files, -f Glob pattern used to filter which files in the path are treated as

migrations. May be specified multiple times, in which case files matching

any of the globs will be treated as migrations. [default: "*.sql"]

--encoding, -e Encoding to use when reading files. [default: "utf8"]

--case-sensitive, -c If set, the glob pattern will be matched against files as a case-sensitive

pattern.

--recursive, -r If set, subdirectories within the path will be traversed and searched for

migrations matching the filter pattern; this option is ignored if the

files glob option contains a slash character.

--order, -o Governs whether migrations are ordered based on the file's basename alone,

or the full file path; must be one of: basename, path [default: "path"]

--logging, -l Logging verbosity; must be one of: debug, info, warn, error, silent

[default: "info"]

--on-read-error Governs behavior when an unusual directory read error is encountered; must

be one of: ignore, log, exit [default: "exit"]

--test If set, instead of performing migrations, dbsync will simply log any

messages about actions it would have performed normally.

--autocommit If set, the commands in the migrations will be run and committed

individually, rather than wrapping each migration inside a single

transaction; this is useful if you want to manually manage multiple

transactions within a migration, or if you want to execute commands not

allowed within a transaction (like DROP DATABASE). This option conflicts

with --migration-at-once and --one-transaction.

--migration-at-once If set, dbsync will not count lines or commands, but instead will load

each migration entirely into memory and pass it to the db at once. This

will keep dbsync from processing the text of the migration, but might

require a lot of memory for large migrations. This option conflicts with

--autocommit.

--one-transaction, -1 If set, dbsync will run all the migrations as a single transaction, rather

than one transaction per migration. Migration table initialization, and

updates to the migration table for pending and failed migrations will not

be executed as part of this transaction. This option conflicts with

--autocommit.

--forget If set, dbsync will not record any migrations that it performs during this

run, nor will it create the migrations table if it doesn't yet exist, but

it will still refuse to run migrations that have succeeded previously

(unless --blindly is also used); this is useful for scripting misc db

commands without requiring any additional client tools to be installed.

--blindly If set, dbsync will not restrict the migrations performed to only those

that have not run successfully before; this is useful for rerunning

previously-run scripts during development, or for scripting misc db

commands without requiring any additional client tools to be installed.

--dollar-quoting If set, dbsync will allow dollar-quoted strings as specified by

PostgreSQL. This is not relevant when --migration-at-once is also used.

For more details, see section 4.1.2.4. (Dollar-quoted String Constants) of

http://www.postgresql.org/docs/9.3/static/sql-syntax-lexical.html

--command-buffering This sets the number of SQL commands to buffer before pausing reading from

the file; must be a positive integer. This is a performance-tuning option

and shouldn't need to be altered for most use cases. [default: 4]

--stack-traces If set, stack traces will be logged with any errors (when present).

--help, -h Show help

--path, -p Directory to scan for migration files; this part of the file path

will not be used when determining whether a migration has been run

before. [required]

--client, --db A db client string, as appropriate to pass to knex in the

initialization object: http://knexjs.org/#Installation-client

[required]

--connection, --conn Additional db connection options, as appropriate to pass to knex in

the initialization object: http://knexjs.org/#Installation-client In

order to set subproperties, use dot notation as in these

examples:

--client=pg --connection=postgres://usr:pw@host:port/dbname?opts

--client=mysql --connection.user=usr --connection.host=localhost

--client=sqlite3 --connection.filename=./mydb.sqlite [required]

--table, -t Table name for tracking migrations [default: "dbsync_migrations"]

--files Glob pattern used to filter which files in the path are treated as

migrations. May be specified multiple times, in which case files

matching any of the globs will be treated as migrations.

[default: "*.sql"]

--encoding, -e Encoding to use when reading files. [default: "utf8"]

--case-sensitive, -c If set, the glob pattern will be matched against files as a

case-sensitive pattern.

--recursive, -r If set, subdirectories within the path will be traversed and searched

for migrations matching the filter pattern; this option is ignored if

the files glob option contains a slash character.

--order, -o Governs whether migrations are ordered based on the file's basename

alone, or the full file path; must be one of: basename, path

[default: "path"]

--logging, -l Logging verbosity; must be one of: debug, info, warn, error, silent

[default: "info"]

--on-read-error Governs behavior when an unusual directory read error is encountered;

must be one of: ignore, log, exit [default: "exit"]

--test If set, instead of performing migrations, dbsync will simply log any

messages about actions it would have performed normally.

--autocommit If set, the commands in the migrations will be run and committed

individually, rather than wrapping each migration inside a single

transaction; this is useful if you want to manually manage multiple

transactions within a migration, or if you want to execute commands

not allowed within a transaction (like DROP DATABASE). This option

conflicts with --migration-at-once and --one-transaction.

--migration-at-once If set, dbsync will not count lines or commands, but instead will

load each migration entirely into memory and pass it to the db at

once. This will keep dbsync from processing the text of the

migration, but might require a lot of memory for large migrations.

This option conflicts with --autocommit.

--one-transaction, -1 If set, dbsync will run all the migrations as a single transaction,

rather than one transaction per migration. Migration table

initialization, and updates to the migration table for pending and

failed migrations will not be executed as part of this transaction.

This option conflicts with --autocommit.

--forget If set, dbsync will not record any migrations that it performs during

this run, nor will it create the migrations table if it doesn't yet

exist, but it will still refuse to run migrations that have succeeded

previously (unless --blindly is also used); this is useful for

scripting misc db commands without requiring any additional client

tools to be installed.

--blindly If set, dbsync will not restrict the migrations performed to only

those that have not run successfully before; this is useful for

rerunning previously-run scripts during development, or for scripting

misc db commands without requiring any additional client tools to be

installed.

--reminder Some environments where dbsync may run (e.g. CircleCI) require

periodic output to ensure the process is running properly. Sometimes

a migration could take a long time, but without this indicating a

problem. In such cases, you can use --reminder to request reminder

output every X minutes (a best-effort attempt is made, so to be safe

you should set a lower value than you really need); a value of 0

disables reminder outout. Value must be non-negative, but can be a

decimal (e.g. --reminder .5) [default: 0]

--dollar-quoting If set, this will force dbsync to allow dollar-quoted strings as

specified by PostgreSQL. The negation, --no-dollar-quoting, is also

available to force quoting to be turned off (this may make a minor

improvement to resource usage by dbsync). The default value for this

option is true when --client is set to 'pg', and false otherwise.

This is not relevant when --migration-at-once is also used. For more

details on dollar-quoting, see section 4.1.2.4 (Dollar-quoted String

Constants) of

http://www.postgresql.org/docs/9.3/static/sql-syntax-lexical.html

--command-buffering This sets the number of SQL commands to buffer before pausing reading

from the file; must be a positive integer. This is a

performance-tuning option and shouldn't need to be altered for most

use cases. [default: 4]

--stack-traces If set, stack traces will be logged with any errors (when present).

```

@@ -149,3 +168,3 @@

* For options that take values, the same allowed values and defaults apply.

* Options allowing dot notation on the command line (such as `--connection`) correspond a nested object

* Options allowing dot notation on the command line (such as `--connection`) correspond to a nested object

* Multiple uses of an option (such as `--files`) correspond to an array of values

@@ -172,3 +191,4 @@

"case-sensitive": true,

"stack-traces": false // all booleans default to false, so this is the same as not specifying it

"stack-traces": false // all booleans (except 'dollar-quoting') default to

// false, so this is the same as not specifying it

}

@@ -183,4 +203,4 @@ ```

* `logging`: passing a falsy value is equivalent to 'silent'

* `logging`: passing an object with functions for its debug, info, warn, and error properties will cause dbsync to use

the passed object for logging

* `logging`: passing an object with functions for its debug, info, warn, error, and log properties will cause dbsync to

use the passed object for logging

@@ -195,3 +215,3 @@ #### Using a Migrator instance

which will be considered for migration, sorted in the order they will be considered according to the `order` option.

Files representing migrations which have already successfully executed will *not* be filtered from this list.

Files representing migrations which have already successfully executed will *not* already be filtered from this list.

* `migrator.shouldMigrationRun(migrationId)` returns a promise of a boolean representing whether the migration needs to

@@ -202,14 +222,13 @@ run based on the migrator's options (so this function will always yield `true` when `blindly: true` is set). The

* `migrator.doAllMigrationsIfNeeded()` is what is used to execute migrations based on a command line invocation. If

`migrator.executionContext.files` is falsy, it will be set with the value returned by `migrator.scanForFiles()`, then

`migrator.executionContext.files` will be used as its list of migration files. This means it is possible to set a

custom list of files (or a custom ordering of files) by setting `migrator.executionContext.files` before calling

`migrator.doAllMigrationsIfNeeded()`. This method returns a promise which resolves to `migrator.executionContext` when

all the migrations in the list have been skipped (based on the result of `migrator.shouldMigrationRun(migrationId)`)

or have succeeded, or when a single migration fails.

* `migrator.doSingleMigrationIfNeeded(migrationId, [migrationSource])` is similar to

`migrator.doAllMigrationsIfNeeded()`, but only performs a single migration (or skips it, based on

`migrator.shouldMigrationRun(migrationId)`), and has a number of advanced behaviors available based on

`migrationSource`:

* if `migrationSource` is `undefined` or `null`, then `migrationId` is treated as a filename; otherwise, it

`migrationId` is a user-defined string id which will identify this migration.

`executionContext.files` is falsy, it will be set with the value returned by `scanForFiles()`, then

`executionContext.files` will be used as its list of migration files. This means it is possible to set a custom list

of files (or a custom ordering of files) by setting `executionContext.files` before calling

`doAllMigrationsIfNeeded()`. This method returns a promise which resolves to `executionContext` when all the migrations

in the list have been skipped (based on the result of `shouldMigrationRun()`) or have succeeded, or when a single

migration fails.

* `migrator.doSingleMigrationIfNeeded(migrationId, [migrationSource])` is similar to `doAllMigrationsIfNeeded()`, but

only performs a single migration (or skips it, based on `shouldMigrationRun()`), and has a number of advanced behaviors

available based on `migrationSource`:

* if `migrationSource` is `undefined` or `null`, then `migrationId` is treated as a filename to load as the

migration; otherwise, `migrationId` should be any user-defined string id which will identify this migration.

* if `migrationSource` is a string, the string will be used as the content of the migration.

@@ -219,7 +238,7 @@ * if `migrationsSource` is an instance of `stream.Readable`, the data from the stream will be used as the content of

can make it return strings by calling `myStream.setEncoding(encoding)`.)

* if `migrationSource` is a promise (or then-able) which resolves to a string or a readable stream, the resolved value

will be used as described above.

* if `migrationSource` is a promise (or then-able) which resolves to a string or a readable stream, the resolved

value will be used as described above.

* if `migrationSource` is a function, the function will be called and its return value (which must be a string,

readable stream, or promise to one of those) will be used as described above. Note that the function will

only be called if `migrator.shouldMigrationRun(migrationId)` resolved to `true` for `migrationId`; this makes it useful

only be called if `shouldMigrationRun()` resolved to `true` for `migrationId`; this makes it useful

for migrations that require some effort/time/resources to set up, such as creating a stream that downloads a file from

@@ -226,0 +245,0 @@ a server. By putting such setup in a function and passing that as the `migrationSource`, the download will never be

		{
		"name": "dbsync",
		"version": "1.0.3",
		"version": "1.1.0",
		"description": "A flexible, easy-to-use, unopinionated schema migration / database change management tool.",
		@@ -5,0 +5,0 @@ "author": "RealtMapster, LLC",

dbsync - npm Package Compare versions

New alerts

Fixed alerts

Improved metrics