better-sqlite3 - npm Package Compare versions

Comparing version 5.3.0 to 5.4.0

docs/api.md

@@ -10,2 +10,3 @@ # API
 - [Database#checkpoint()](#checkpointdatabasename---this)
+- [Database#backup()](#backupdestination-options---promise)
 - [Database#function()](#functionname-options-function---this)
@@ -32,5 +33,7 @@ - [Database#aggregate()](#aggregatename-options---this)
 
+- `options.verbose`: provide a function that gets called with every SQL string executed by the database connection (default: `null`).
+
 ```js
 const Database = require('better-sqlite3');
-const db = new Database('foobar.db', { readonly: true });
+const db = new Database('foobar.db', { verbose: console.log });
 ```
@@ -132,2 +135,37 @@
 
+### .backup(*destination*, [*options*]) -> *promise*
+
+Initiates a [backup](https://www.sqlite.org/backup.html) of the database, returning a [promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Using_promises) for when the backup is complete. If the backup fails, the promise will be rejected with an `Error`. You can optionally backup an attached database by setting the `attached` option to the name of the desired attached database.
+
+```js
+db.backup(`backup-${Date.now()}.db`)
+  .then(() => {
+    console.log('backup complete!');
+  })
+  .catch((err) => {
+    console.log('backup failed:', err);
+  });
+```
+
+You can continue to use the database normally while a backup is in progress. If the same database connection mutates the database while performing a backup, those mutations will be reflected in the backup automatically. However, if a *different* connection mutates the database during a backup, the backup will be forcefully restarted. Therefore, it's recommended that only a single connection is responsible for mutating the database if online backups are being performed.
+
+You can monitor the progress of the backup by setting the `progress` option to a callback function. That function will be invoked every time the backup makes progress, providing an object with two properties:
+
+- `.totalPages`: the total number of pages in the source database (and thus, the number of pages that the backup will have when completed) at the time of this progress report.
+- `.remainingPages`: the number of pages that still must be transferred before the backup is complete.
+
+By default, `100` [pages](https://www.sqlite.org/fileformat.html#pages) will be transferred after each cycle of the event loop. However, you can change this setting as often as you like by returning a number from the `progress` callback. You can even return `0` to effectively pause the backup altogether. In general, the goal is to maximize throughput while reducing pause times. If the transfer size is very low, pause times will be low, but it may take a while to complete the backup. On the flip side, if the setting is too high, pause times will be greater, but the backup might complete sooner. In most cases, `100` has proven to be a strong compromise, but the best setting is dependent on your computer's specifications and the nature of your program. Do not change this setting without measuring the effectiveness of your change. You should not assume that your change will even have the intended effect, unless you measure it for your unique situation.
+
+If the `progress` callback throws an exception, the backup will be aborted. Usually this happens due to an unexpected error, but you can also use this behavior to voluntarily cancel the backup operation. If the parent database connection is closed, all pending backups will be automatically aborted.
+
+```js
+let paused = false;
+db.backup(`backup-${Date.now()}.db`, {
+  progress({ totalPages: t, remainingPages: r }) {
+    console.log(`progress: ${((t - r) / t * 100).toFixed(1)}%`);
+    return paused ? 0 : 200;
+  }
+});
+```
+
 ### .function(*name*, [*options*], *function*) -> *this*
@@ -236,5 +274,5 @@
 process.on('exit', () => db.close());
-process.on('SIGINT', () => db.close());
-process.on('SIGHUP', () => db.close());
-process.on('SIGTERM', () => db.close());
+process.on('SIGHUP', () => process.exit(128 + 1));
+process.on('SIGINT', () => process.exit(128 + 2));
+process.on('SIGTERM', () => process.exit(128 + 15));
 ```
@@ -241,0 +279,0 @@

docs/compilation.md

@@ -20,5 +20,5 @@ # Custom configuration
 SQLITE_OMIT_GET_TABLE
-SQLITE_OMIT_TRACE
 SQLITE_OMIT_TCL_VARIABLE
 SQLITE_OMIT_PROGRESS_CALLBACK
+SQLITE_TRACE_SIZE_LIMIT=32
 SQLITE_DEFAULT_CACHE_SIZE=-16000
@@ -25,0 +25,0 @@ SQLITE_DEFAULT_FOREIGN_KEYS=1

lib/database.js

@@ -22,2 +22,3 @@ 'use strict';
 	const timeout = 'timeout' in options ? options.timeout : 5000;
+	const verbose = 'verbose' in options ? options.verbose : null;
 
@@ -28,2 +29,3 @@ if (readonly && (memory || anonymous)) throw new TypeError('In-memory databases cannot be readonly');
 	if (timeout > 0x7fffffff) throw new RangeError('Option "timeout" cannot be greater than 2147483647');
+	if (verbose != null && typeof verbose !== 'function') throw new TypeError('Expected the "verbose" option to be a function');
 
@@ -45,3 +47,3 @@ if (!memory && !anonymous && !fs.existsSync(path.dirname(filename))) {
 	}
-	return new CPPDatabase(filename, filenameGiven, memory || anonymous, readonly, fileMustExist, timeout);
+	return new CPPDatabase(filename, filenameGiven, memory || anonymous, readonly, fileMustExist, timeout, verbose || null);
 }
@@ -52,2 +54,3 @@
 util.wrap(CPPDatabase, 'aggregate', require('./aggregate'));
+util.wrap(CPPDatabase, 'backup', require('./backup'));
 CPPDatabase.prototype.transaction = require('./transaction');
@@ -54,0 +57,0 @@ CPPDatabase.prototype.constructor = Database;

package.json

 {
   "name": "better-sqlite3",
-  "version": "5.3.0",
+  "version": "5.4.0",
   "description": "The fastest and simplest library for SQLite3 in Node.js.",
@@ -5,0 +5,0 @@ "homepage": "http://github.com/JoshuaWise/better-sqlite3",

README.md

@@ -11,2 +11,9 @@ # better-sqlite3 [![Build Status](https://travis-ci.org/JoshuaWise/better-sqlite3.svg?branch=master)](https://travis-ci.org/JoshuaWise/better-sqlite3)
 
+## Help this project stay strong! 💪
+
+`better-sqlite3` is used by thousands of developers and engineers on a daily basis. Long nights and weekends were spent keeping this project strong and dependable, with no ask for compensation or funding, until now. If your company uses `better-sqlite3`, ask your manager to consider supporting the project:
+
+- [Become a backer on Patreon](https://www.patreon.com/joshuawise)
+- [Make a one-time donation on PayPal](https://www.paypal.me/joshuathomaswise)
+
 ## How other libraries compare
@@ -20,3 +27,2 @@
 > You can verify these results by [running the benchmark yourself](./docs/benchmark.md).
-> *Both [npm/sqlite](https://www.npmjs.com/package/sqlite) and [npm/sqlite3](https://www.npmjs.com/package/sqlite3) have nearly identical performance because they both use the [same engine](https://github.com/mapbox/node-sqlite3).*
 
@@ -23,0 +29,0 @@ ## Installation

test/10.database.open.js

@@ -40,2 +40,3 @@ 'use strict';
 			expect(existsSync('[object Object]')).to.be.false;
+			db.close();
 		}
@@ -51,2 +52,3 @@ });
 		expect(existsSync(':memory:')).to.be.false;
+		db.close();
 	});
@@ -62,2 +64,3 @@ it('should allow named in-memory databases to be created', function () {
 		expect(existsSync(util.current())).to.be.false;
+		db.close();
 	});
@@ -73,2 +76,3 @@ it('should allow disk-bound databases to be created', function () {
 		expect(existsSync(util.current())).to.be.true;
+		db.close();
 	});
@@ -93,2 +97,3 @@ it('should not allow conflicting in-memory options', function () {
 		expect(existsSync(util.current())).to.be.true;
+		db.close();
 	});
@@ -114,2 +119,3 @@ it('should not allow the "readonly" option for in-memory databases', function () {
 		expect(existsSync(util.current())).to.be.true;
+		db.close();
 	});
@@ -138,6 +144,9 @@ it('should accept the "timeout" option', function () {
 		expect(() => new Database(util.current(), { timeout: 0x80000000 })).to.throw(RangeError);
+		blocker.close();
 	});
-	it('should throw an Error if opening the database failed', function () {
+	it('should throw an Error if the directory does not exist', function () {
 		expect(existsSync(util.next())).to.be.false;
-		expect(() => new Database(`temp/nonexistent/abcfoobar123/${util.current()}`)).to.throw(TypeError);
+		const filepath = `temp/nonexistent/abcfoobar123/${util.current()}`;
+		expect(() => new Database(filepath)).to.throw(TypeError);
+		expect(existsSync(filepath)).to.be.false;
 		expect(existsSync(util.current())).to.be.false;
@@ -153,3 +162,4 @@ });
 		expect(Database.prototype).to.equal(Object.getPrototypeOf(db));
+		db.close();
 	});
 });

test/13.database.prepare.js

@@ -11,3 +11,4 @@ 'use strict';
 	});
-	const assertStmt = (stmt, source, db, reader) => {
+
+	function assertStmt(stmt, source, db, reader) {
 		expect(stmt.source).to.equal(source);
@@ -18,3 +19,3 @@ expect(stmt.constructor.name).to.equal('Statement');
 		expect(() => new stmt.constructor(source)).to.throw(TypeError);
-	};
+	}
 
@@ -21,0 +22,0 @@ it('should throw an exception if a string is not provided', function () {

test/23.statement.iterate.js

@@ -122,15 +122,2 @@ 'use strict';
 	});
-	it('should not be able to invoke .pluck() while the database is busy', function () {
-		const stmt1 = this.db.prepare("SELECT * FROM entries ORDER BY rowid");
-		const stmt2 = this.db.prepare("SELECT * FROM entries ORDER BY rowid LIMIT 2");
-		let i = 0;
-		for (const data of stmt1.iterate()) {
-			++i;
-			expect(() => stmt1.pluck()).to.throw(TypeError);
-			expect(() => stmt2.pluck()).to.throw(TypeError);
-			expect(() => stmt1.pluck(false)).to.throw(TypeError);
-			expect(() => stmt2.pluck(false)).to.throw(TypeError);
-		}
-		expect(i).to.equal(10);
-	});
 	it('should close the iterator when throwing in a for-of loop', function () {
@@ -170,43 +157,2 @@ const err = new Error('foobar');
 	});
-	it('should not allow other database operations to execute while open', function () {
-		const stmt1 = this.db.prepare('SELECT * FROM entries ORDER BY rowid');
-		const stmt2 = this.db.prepare('CREATE TABLE numbers (number INTEGER)');
-		let count = 0;
-		for (const row of this.db.prepare('SELECT * FROM entries ORDER BY rowid').iterate()) {
-			++count;
-			expect(() => this.db.close()).to.throw(TypeError);
-			expect(() => this.db.pragma('cache_size')).to.throw(TypeError);
-			expect(() => this.db.checkpoint()).to.throw(TypeError);
-			expect(() => this.db.prepare('SELECT * FROM entries ORDER BY rowid')).to.throw(TypeError);
-			expect(() => this.db.transaction(() => {})).to.throw(TypeError);
-			expect(() => stmt1.get()).to.throw(TypeError);
-			expect(() => stmt1.all()).to.throw(TypeError);
-			expect(() => stmt1.iterate()).to.throw(TypeError);
-			expect(() => stmt2.run()).to.throw(TypeError);
-		}
-		expect(count).to.equal(10);
-	});
-	it('should allow database operations after closing the iterator', function () {
-		const row = { a: 'foo', b: 1, c: 3.14, d: Buffer.alloc(4).fill(0xdd), e: null };
-		const stmt = this.db.prepare("SELECT * FROM entries ORDER BY rowid");
-		this.db.prepare('SELECT 555');
-		const iterator = stmt.iterate();
-		expect(() => this.db.prepare('SELECT 555')).to.throw(TypeError);
-		expect(iterator.next()).to.deep.equal({ value: row, done: false });
-		row.b += 1;
-		expect(() => this.db.prepare('SELECT 555')).to.throw(TypeError);
-		expect(iterator.next()).to.deep.equal({ value: row, done: false });
-		row.b += 1;
-		expect(() => this.db.prepare('SELECT 555')).to.throw(TypeError);
-		expect(iterator.next()).to.deep.equal({ value: row, done: false });
-		expect(() => this.db.prepare('SELECT 555')).to.throw(TypeError);
-		expect(iterator.return()).to.deep.equal({ value: undefined, done: true });
-		this.db.prepare('SELECT 555');
-		expect(iterator.next()).to.deep.equal({ value: undefined, done: true });
-		this.db.prepare('SELECT 555');
-		expect(iterator.return()).to.deep.equal({ value: undefined, done: true });
-		this.db.prepare('SELECT 555');
-		expect(iterator.next()).to.deep.equal({ value: undefined, done: true });
-		this.db.prepare('SELECT 555');
-	});
 	it('should accept bind parameters', function () {
@@ -213,0 +159,0 @@ const shouldHave = (SQL, desiredData, args) => {

test/25.statement.columns.js

@@ -13,17 +13,2 @@ 'use strict';
 
-	it('should throw an exception if the database is closed', function () {
-		const stmt = this.db.prepare('SELECT 5.0 as d, * FROM entries');
-		this.db.close();
-		expect(() => stmt.columns()).to.throw(TypeError);
-	});
-	it('should throw an exception if the database is busy', function () {
-		this.db.prepare('INSERT INTO entries VALUES (?, ?, ?)').run('foo', 5, Buffer.from('bar'));
-		const stmt = this.db.prepare('SELECT 5.0 as d, * FROM entries');
-		let invoked = false;
-		for (const row of this.db.prepare(stmt.source).iterate()) {
-			expect(() => stmt.columns()).to.throw(TypeError);
-			invoked = true;
-		}
-		expect(invoked).to.be.true;
-	});
 	it('should throw an exception if invoked on a non-reader statement', function () {
@@ -30,0 +15,0 @@ const stmt = this.db.prepare('INSERT INTO entries VALUES (?, ?, ?)');

test/31.database.checkpoint.js

@@ -15,2 +15,7 @@ 'use strict';
 	});
+	after(function () {
+		db1.close();
+		db2.close();
+	});
+
 	function fillWall(count, expectation) {
@@ -17,0 +22,0 @@ [db1, db2].forEach((db) => {

test/32.database.function.js

@@ -99,35 +99,2 @@ 'use strict';
 	});
-	it('should throw an exception if the database is busy', function () {
-		let ranOnce = false;
-		for (const x of this.db.prepare('SELECT 2').pluck().iterate()) {
-			expect(x).to.equal(2);
-			ranOnce = true;
-			expect(() => this.db.function('fn', () => {})).to.throw(TypeError);
-		}
-		expect(ranOnce).to.be.true;
-		this.db.function('fn', () => {});
-	});
-	it('should cause the database to become busy when executing the function', function () {
-		let ranOnce = false;
-		this.db.function('a', () => {});
-		this.db.function('b', () => {
-			ranOnce = true;
-			expect(() => this.db.exec('SELECT a()')).to.throw(TypeError);
-			expect(() => this.db.prepare('SELECT 555')).to.throw(TypeError);
-			expect(() => this.db.pragma('cache_size')).to.throw(TypeError);
-			expect(() => this.db.function('z', () => {})).to.throw(TypeError);
-		});
-
-		expect(this.get('b()')).to.equal(null);
-		expect(ranOnce).to.be.true;
-		ranOnce = false;
-
-		expect(this.db.exec('SELECT b()')).to.equal(this.db);
-		expect(ranOnce).to.be.true;
-
-		this.db.exec('SELECT a()')
-		this.db.prepare('SELECT 555');
-		this.db.pragma('cache_size');
-		this.db.function('zz', () => {});
-	});
 	it('should cause the function to throw when returning an invalid value', function () {
@@ -169,3 +136,3 @@ this.db.function('fn', x => ({}));
 				total += value;
-				expect(() => this.db.prepare('SELECT fn(value) FROM iterable')).to.throw(TypeError);
+				expect(() => this.db.exec('SELECT fn(value) FROM iterable LIMIT 4')).to.throw(TypeError);
 			}
@@ -176,4 +143,7 @@ }).to.throw(err);
 		expect(iterator.next()).to.deep.equal({ value: undefined, done: true });
-		this.db.prepare('SELECT fn(value) FROM iterable').pluck().iterate().return();
 		expect(total).to.equal(1 + 2 + 4 + 8);
+
+		i = 0;
+		this.db.exec('SELECT fn(value) FROM iterable LIMIT 4');
+		expect(i).to.equal(4);
 	});
@@ -212,3 +182,3 @@ it('should be able to register multiple functions with the same name', function () {
 			this.db.function('fn', (x) => {
-				expect(() => this.db.prepare('SELECT 555')).to.throw(TypeError);
+				expect(() => this.db.exec('SELECT 555')).to.throw(TypeError);
 				return x * 2;
@@ -220,6 +190,6 @@ });
 				expect(x).to.equal(1110);
-				expect(() => this.db.prepare('SELECT 555')).to.throw(TypeError);
+				expect(() => this.db.exec('SELECT 555')).to.throw(TypeError);
 			}
 			expect(ranOnce).to.be.true;
-			this.db.prepare('SELECT 555');
+			this.db.exec('SELECT 555');
 		});
@@ -226,0 +196,0 @@ specify('was_js_error state', function () {

test/33.database.aggregate.js

@@ -423,3 +423,3 @@ 'use strict';
 				total += value;
-				expect(() => this.db.prepare('SELECT wn(value) OVER (ROWS CURRENT ROW) FROM iterable')).to.throw(TypeError);
+				expect(() => this.db.exec('SELECT wn(value) OVER (ROWS CURRENT ROW) FROM iterable LIMIT 4')).to.throw(TypeError);
 			}
@@ -430,4 +430,7 @@ }).to.throw(err);
 		expect(iterator.next()).to.deep.equal({ value: undefined, done: true });
-		this.db.prepare('SELECT wn(value) OVER (ROWS CURRENT ROW) FROM iterable').pluck().iterate().return();
 		expect(total).to.equal(1 + 2 + 4 + 8);
+
+		i = 0;
+		this.db.exec('SELECT wn(value) OVER (ROWS CURRENT ROW) FROM iterable LIMIT 4');
+		expect(i).to.equal(4);
 	});
@@ -570,3 +573,3 @@ it('should be able to register multiple aggregates with the same name', function () {
 			this.db.aggregate('agg', { step: (ctx, x) => {
-				expect(() => this.db.prepare('SELECT 555')).to.throw(TypeError);
+				expect(() => this.db.exec('SELECT 555')).to.throw(TypeError);
 				return x * 2 + ctx;
@@ -578,6 +581,6 @@ } });
 				expect(x).to.equal(1110);
-				expect(() => this.db.prepare('SELECT 555')).to.throw(TypeError);
+				expect(() => this.db.exec('SELECT 555')).to.throw(TypeError);
 			}
 			expect(ranOnce).to.be.true;
-			this.db.prepare('SELECT 555');
+			this.db.exec('SELECT 555');
 		});
@@ -584,0 +587,0 @@ specify('was_js_error state', function () {

test/34.database.load-extension.js

@@ -21,6 +21,2 @@ 'use strict';
 	});
-	it('should throw an exception if the database is closed', function () {
-		this.db.close();
-		expect(() => this.db.loadExtension(filepath)).to.throw(TypeError);
-	});
 	it('should throw an exception if the database is busy', function () {
@@ -27,0 +23,0 @@ let invoked = false;

test/40.integers.js

@@ -103,19 +103,2 @@ 'use strict';
 	});
-	it('should forbid invoking .safeIntegers() while the database is busy', function () {
-		const int = Integer.fromBits(4243423, 234234234);
-		this.db.prepare('INSERT INTO entries VALUES (?, ?, ?)').run(int, int, int);
-		this.db.prepare('INSERT INTO entries VALUES (?, ?, ?)').run(int, int, int);
-
-		let ranOnce = false;
-		const stmt1 = this.db.prepare('SELECT * FROM entries LIMIT 10');
-		const stmt2 = this.db.prepare('INSERT INTO entries VALUES (?, ?, ?)');
-		for (const row of stmt1.iterate()) {
-			ranOnce = true;
-			expect(() => stmt1.safeIntegers()).to.throw(TypeError);
-			expect(() => stmt2.safeIntegers()).to.throw(TypeError);
-			expect(() => stmt1.safeIntegers(false)).to.throw(TypeError);
-			expect(() => stmt2.safeIntegers(false)).to.throw(TypeError);
-		}
-		expect(ranOnce).to.be.true;
-	});
 });

test/41.at-exit.js

@@ -7,2 +7,3 @@ 'use strict';
 	this.slow(500);
+
 	const source = (filename1, filename2) => `
@@ -9,0 +10,0 @@ 'use strict';

test/50.misc.js

@@ -5,41 +5,41 @@ 'use strict';
 describe('miscellaneous', function () {
-  beforeEach(function () {
-    this.db = new Database(util.next());
-  });
-  afterEach(function () {
-    this.db.close();
-  });
+	beforeEach(function () {
+		this.db = new Database(util.next());
+	});
+	afterEach(function () {
+		this.db.close();
+	});
 
-  it('persists non-trivial quantities of reads and writes', function () {
-    const runDuration = 1000;
-    const runUntil = Date.now() + runDuration;
-    this.slow(runDuration * 10);
-    this.timeout(runDuration * 3);
-    this.db.pragma("journal_mode = WAL");
-    this.db.prepare("CREATE TABLE foo (a INTEGER, b TEXT, c REAL)").run();
+	it('persists non-trivial quantities of reads and writes', function () {
+		const runDuration = 1000;
+		const runUntil = Date.now() + runDuration;
+		this.slow(runDuration * 10);
+		this.timeout(runDuration * 3);
+		this.db.pragma("journal_mode = WAL");
+		this.db.prepare("CREATE TABLE foo (a INTEGER, b TEXT, c REAL)").run();
 
-    let i = 1;
-    const r = 0.141592654;
-    const insert = this.db.prepare("INSERT INTO foo VALUES (?, ?, ?)");
-    const insertMany = this.db.transaction((count) => {
-      for (const end = i + count; i < end; ++i) {
-        expect(insert.run(i, String(i), i + r))
-          .to.deep.equal({ changes: 1, lastInsertRowid: i });
-      }
-    });
+		let i = 1;
+		const r = 0.141592654;
+		const insert = this.db.prepare("INSERT INTO foo VALUES (?, ?, ?)");
+		const insertMany = this.db.transaction((count) => {
+			for (const end = i + count; i < end; ++i) {
+				expect(insert.run(i, String(i), i + r))
+					.to.deep.equal({ changes: 1, lastInsertRowid: i });
+			}
+		});
 
-    // Batched transactions of 100 inserts.
-    while (Date.now() < runUntil) insertMany(100);
+		// Batched transactions of 100 inserts.
+		while (Date.now() < runUntil) insertMany(100);
 
-    // Expect 10K~50K on reasonable machines.
-    expect(i).to.be.above(1000);
+		// Expect 10K~50K on reasonable machines.
+		expect(i).to.be.above(1000);
 
-    const select = this.db.prepare("SELECT * FROM foo ORDER BY a DESC");
-    for (const row of select.iterate()) {
-      i -= 1;
-      expect(row).to.deep.equal({ a: i, b: String(i), c: i + r });
-    }
+		const select = this.db.prepare("SELECT * FROM foo ORDER BY a DESC");
+		for (const row of select.iterate()) {
+			i -= 1;
+			expect(row).to.deep.equal({ a: i, b: String(i), c: i + r });
+		}
 
-    expect(i).to.equal(1);
-  });
+		expect(i).to.equal(1);
+	});
 });

New alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Fixed alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Improved metrics

Total package byte prevSize

2607494

increased by2.53%

Number of package files

80

increased by6.67%

Lines of code

3739

increased by25.89%

Number of lines in readme file

73

increased by8.96%

Worsened metrics

Number of low supply chain risk alerts

13

increased by18.18%

No dependency changes