dbay - npm Package Compare versions

Comparing version 0.1.0 to 0.100.3

package.json

 {
   "name": "dbay",
-  "version": "0.1.0",
+  "version": "0.100.3",
   "description": "In-Process, In-Memory & File-Based Relational Data Processing with SQLite, BetterSQLite3",
   "main": "lib/main.js",
-  "scripts": {
-    "build": "coffee --map -o lib -c src",
-    "test": "echo see 'https://github.com/loveencounterflow/hengist/tree/master/dev/dbay'",
-    "preinstall": "./build-sqlite3"
-  },
   "repository": {
@@ -30,4 +25,10 @@ "type": "git",
     "guy": "^2.2.0",
-    "intertype": "7.6.7"
+    "intertype": "7.6.7",
+    "temp": "^0.9.4"
+  },
+  "scripts": {
+    "build": "coffee --map -o lib -c src",
+    "test": "echo see 'https://github.com/loveencounterflow/hengist/tree/master/dev/dbay'",
+    "preinstall": "./build-sqlite3"
   }
-}
+}

README-construction.md

@@ -8,55 +8,63 @@
 
-- [All Parameters in Systematic Order](#all-parameters-in-systematic-order)
-- [Valid Parameter Combinations](#valid-parameter-combinations)
-- [Faulty Parameter Combinations](#faulty-parameter-combinations)
+- [Using Defaults](#using-defaults)
+- [Using Parameters](#using-parameters)
+  - [All Parameters in Systematic Order](#all-parameters-in-systematic-order)
+  - [Valid Parameter Combinations](#valid-parameter-combinations)
+  - [Invalid Parameter Combinations](#invalid-parameter-combinations)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 
-* In order to construct (instantiate) a DBay object, you can call the constructor without any arguments:
+### Using Defaults
 
-  ```coffee
-  { Dbay }  = require 'dbay'
-  db        = new Dbay()
-  ```
+In order to construct (instantiate) a DBay object, you can call the constructor without any arguments:
 
-  The `db` object will then have a property `db.sqlt` that is a `better-sqlite3` connection to an
-  in-memory DB (a RAM DB in our terminology).
+```coffee
+{ Dbay }  = require 'dbay'
+db        = new Dbay()
+```
 
-* You can also call the constructor with a configuration object that may have one or more of the following
-  fields:
+The `db` object will then have two properties `db.sqlt1` and `db.sqlt2` that are `better-sqlite3`
+connections to the same in-memory DB (a RAM DB in our terminology). This is achieved by passing an URL like
+`file:_6200294332?mode=memory&cache=shared` to `better-sqlite3` where the name (`_6200294332`) is a random
+10-digit number.
 
-  * **`cfg.ram`** (`?boolean`): Specifies whether a RAM DB is to be opened. All DBay in-memory DBs are named
-    so several connections to the same RAM DB can be opened (this is necessitated by a <del>shortcome</del>
-    <ins>feature</ins> of `better-sqlite3` that prohibits any reads against the DB from within User-Defined
-    Functions).
+### Using Parameters
 
-    * When neither **`cfg.path`** nor **`cfg.dbnick`** are given, an empty RAM DB will be opened.
-    * When **`cfg.dbnick`** (but not **`cfg.path`**) is given, a
-    * When **`cfg.path`** is given, an SQLite DB file will be (created if non-existant and) opened; then,
-      the DB will be mirrored to RAM so now you have a RAM DB associated with a disk location. You can use
-      `db.save()` any number of times to write changes to disk. DB contents will be lost should the
-      process terminate after changes to the DB but before `db.save()` terminates. This mode of operation
-      is called 'Eventual Persistency'.
+You can also call the constructor with a configuration object that may have one or more of the following
+fields:
 
-  * **`cfg.path`** (`?non-empty text`): Specifies which file system path to save the DB to.
-    * When `path` is given but `ram` is not set (or `null` or `undefined`), `ram` will assume the value
-      `false`.
-    * When `path` is given
-      * and `ram` is **`false`**, a file DB will be opened from or created at the location given. This DB
-        will have Continuous Persistency, i.e. operate in the normal DB mode where all changes are reflected
-        on disk and thus made durable with a high degree of safety against data losses. Otherwise,
-      * when `ram` is `true`, a RAM DB will be opened.
+* **`cfg.ram`** (`?boolean`): Specifies whether a RAM DB is to be opened. All DBay in-memory DBs are named
+  so several connections to the same RAM DB can be opened (this is necessitated by a <del>shortcome</del>
+  <ins>feature</ins> of `better-sqlite3` that prohibits any reads against the DB from within User-Defined
+  Functions).
 
-  * **`cfg.dbnick`** (`?URL-safe word`): name given to a RAM DB. It will be used to construct a URL that
-    will be passed to SQLite. There's little use in passing in `dbnick` explicitly; if one wishes to
-    construct multiple `Dbay()` objects to the same RAM DB, one can always use the `cfg` object of the first
-    instance:
+  * When neither **`cfg.path`** nor **`cfg.dbnick`** are given, an empty RAM DB will be opened.
+  * When **`cfg.dbnick`** (but not **`cfg.path`**) is given, a
+  * When **`cfg.path`** is given, an SQLite DB file will be (created if non-existant and) opened; then,
+    the DB will be mirrored to RAM so now you have a RAM DB associated with a disk location. You can use
+    `db.save()` any number of times to write changes to disk. DB contents will be lost should the
+    process terminate after changes to the DB but before `db.save()` terminates. This mode of operation
+    is called 'Eventual Persistency'.
 
-    ```coffee
-    db1 = new Dbay { ram: true, }
-    db2 = new Dbay db1.cfg
-    ```
+* **`cfg.path`** (`?non-empty text`): Specifies which file system path to save the DB to.
+  * When `path` is given but `ram` is not set (or `null` or `undefined`), `ram` will assume the value
+    `false`.
+  * When `path` is given
+    * and `ram` is **`false`**, a file DB will be opened from or created at the location given. This DB
+      will have Continuous Persistency, i.e. operate in the normal DB mode where all changes are reflected
+      on disk and thus made durable with a high degree of safety against data losses. Otherwise,
+    * when `ram` is `true`, a RAM DB will be opened.
 
-### All Parameters in Systematic Order
+* **`cfg.dbnick`** (`?URL-safe word`): name given to a RAM DB. It will be used to construct a URL that
+  will be passed to SQLite. There's little use in passing in `dbnick` explicitly; if one wishes to
+  construct multiple `Dbay()` objects to the same RAM DB, one can always use the `cfg` object of the first
+  instance:
 
+  ```coffee
+  db1 = new Dbay { ram: true, }
+  db2 = new Dbay db1.cfg
+  ```
+
+#### All Parameters in Systematic Order
+
 **Note** in the below tables, `in.*` parameters are those passed in when calling `new Dbay { ... }`; `out.*`
@@ -96,3 +104,3 @@ parameters are those to be found under `db.cfg.*` in the newly constructed instance. Observe that
 
-### Valid Parameter Combinations
+#### Valid Parameter Combinations
 
@@ -118,3 +126,3 @@ |   nr  |      in.ram     |   in.path   | in.dbnick  | out.ram |   out.path  |    out.dbnick   | out.persistency |
 
-### Faulty Parameter Combinations
+#### Invalid Parameter Combinations
 
@@ -121,0 +129,0 @@ * When a `path` is given, `dbnick` must not be set. In the future, we may allow this `dbnick` to be used when

README.md

@@ -37,3 +37,4 @@ # DBay
 
-* [`Dbay` object construction](./README-construction.md)
+* **[`Dbay` object construction](./README-construction.md)**
+* **[Benchmarks](./README-benchmarks.md)**
 
@@ -65,1 +66,25 @@ ## Note on Package Structure
 * **[–]** at construction time, allow `dbnick` when `path` is given and `ram` is `false`
+* **[–]** to solve the table-UDF-with-DB-access conundrum, consider
+  * <del>**[+]** https://github.com/mapnik/mapnik/issues/797, where connection parameters are discussed (see also
+    https://www.sqlite.org/c3ref/open.html);</del> <ins>nothing of interested AFAICS</ins>
+  * **[–]** mirroring a given DB into a second (RAM or file) location, taking care to replay any goings-on
+    on both instances. This is probably unattractive from a performance POV.
+  * **[–]** using [NodeJS worker threads](https://nodejs.org/api/worker_threads.html) to perform updates;
+    maybe one could even continuously mirror a RAM DB on disk to get a near-synchronous copy, obliviating
+    the necessity to explicitly call `db.save()`. See
+    https://github.com/JoshuaWise/better-sqlite3/blob/master/docs/threads.md
+  * **[–]** implementing **macros** so one could write eg `select * from foo( x ) as d;` to get `select *
+    from ( select a, b, c from blah order by 1 ) as d` (i.e. inline expansion)
+  * **[–]** Obeserve that, seemingly, only *table-valued* UDFs hang while with shared-cache we already *can*
+    issue `select`s from inside UDFs, so maybe there's a teeny, fixable difference between how both are
+    implemented that leads to the undesirable behavior
+* **[–]** let users choose between SQLite-only RAM DBs and `tmpfs`-based in-memory DBs (b/c the latter allow
+  `pragma journal_mode = WAL` for better concurrent access). Cons include: `tmpfs`-based RAM DBs necessitate
+  mounting a RAM disk which needs `sudo` rights, so might as well just instruct users to mount RAM disk,
+  then use that path? Still, it would be preferrable to have some automatic copy-to-durable in place.
+
+
+
+
+
+

No alert changes

Improved metrics

Total package byte prevSize

11855646

increased by0.04%

Number of package files

54

increased by1.89%

Number of lines in readme file

89

increased by39.06%

Worsened metrics

Dependency count

4

increased by33.33%

Dependency changes

• + Addedtemp@^0.9.4

• + Addedbalanced-match@1.0.2(transitive)

• + Addedbrace-expansion@1.1.11(transitive)

• + Addedconcat-map@0.0.1(transitive)

• + Addedfs.realpath@1.0.0(transitive)

• + Addedglob@7.2.3(transitive)

• + Addedinflight@1.0.6(transitive)

• + Addedinherits@2.0.4(transitive)

• + Addedminimatch@3.1.2(transitive)

• + Addedminimist@1.2.8(transitive)

• + Addedmkdirp@0.5.6(transitive)

• + Addedonce@1.4.0(transitive)

• + Addedpath-is-absolute@1.0.1(transitive)

• + Addedrimraf@2.6.3(transitive)

• + Addedtemp@0.9.4(transitive)

• + Addedwrappy@1.0.2(transitive)