
Security News
vlt Launches "reproduce": A New Tool Challenging the Limits of Package Provenance
vlt's new "reproduce" tool verifies npm packages against their source code, outperforming traditional provenance adoption in the JavaScript ecosystem.
@thinkmill/keystone-mosql-yaml-gen
Advanced tools
A basic generator of MoSQL YAML config files for KeystoneJS models
Tools for generating a MoSQL-compatible YAML document describing the lists configured for your Keystone 4 instance. This can be useful for exporting data from a Keystone instance or streaming updates to a reporting database in near real-time.
You'll need to install MoSQL and PostgreSQL separately.
const keystone = require('keystone');
const mosqlYaml = require('@thinkmill/keystone-mosql-yaml-gen');
// Create a generator instance
const yamlGenerator = new mosqlYaml.YamlGenerator(keystone);
// Extract the MoSQL YAML for your lists
const collectionsYaml = yamlGenerator.generateYaml();
console.log(collectionsYaml);
A drop-in ExpressJS endpoint is included. This can be useful if your schema is often changing; a process could be configured to periodically download an up-to-date config.
Add the endpoint to your Express app like this:
const keystone = require('keystone');
const mosqlYaml = require('@thinkmill/keystone-mosql-yaml-gen');
app.get('/api/keystoneListsYaml', mosqlYaml.createMosqlYamlEndpoint(keystone));
Alternatively, create your own endpoint:
const keystone = require('keystone');
const mosqlYaml = require('@thinkmill/keystone-mosql-yaml-gen');
const endpoint = function (req, res, next) {
const yamlGenerator = new mosqlYaml.YamlGenerator(keystone);
const collectionsYaml = yamlGenerator.generateYaml();
res.set('content-disposition', `attachment; filename="${yamlGenerator.getFilename()}"`);
res.set('content-Type', 'application/x-yaml');
res.send(collectionsYaml);
};
The config generated describes the collections and fields to copy.
It's supplied to mosql
like this:
mosql -c 180502-keystone-MoSQL.yaml [--sql postgres://sql-server/sql-db] [--mongo mongodb://mongo-uri]
See the MoSQL docs for more options.
Unlike other database systems, in PostgreSQL, there is no performance difference between varchar
, varchar(n)
and text
types.
We preference text
.
Numbers in JavaScript are 64-bit floating points; equivalent to PGs double precision
type.
Note that for Money
fields are also imported as double precision
.
These values should be converted to a lossless type before being manipulated (eg. numeric(20, 4)
or similar).
JavaScript Date
objects have no timezone information -- they're epoch-base and effectively in UTC.
Importing as timestamp with time zone
will maintaining the correct value while defaulting the time zone stored to UTC.
Date
objects also maintain only millisecond precision (ie. 1/1,000 of a second).
PostgreSQL on the other hand, defaults to microsecond precision for it's timestamp
types (ie. 1/1,000,000 of a second).
This tool sticks with the Postgres default but column types could later be altered to timestamp (3) with time zone
without any data loss.
Currently passwords are intentionally excluded from the dumped fields.
(TODO: Add configuration options for this)
Values are mapped to individual columns where possible:
Keystone Type | PostgreSQL Type | Notes |
---|---|---|
datetime | timestamp with time zone | See Dates above |
date | timestamp with time zone | |
number | double precision | |
relationship | text | When many !== true |
relationship | text array | When many === true |
select | text | |
text | text | |
boolean | boolean | |
code | text | |
email | text | |
html | text | |
markdown | text | |
textarea | text | |
money | double precision | See Numbers above |
geopoint | double precision array | |
textarray | text array |
Some of the more complex, multi-value types are mapped to multiple columns:
name
TypeKeystone Field | Column Name | PostgreSQL Type |
---|---|---|
(key).first | (key)_first | text |
(key).last | (key)_last | text |
location
TypeKeystone Field | Column Name | PostgreSQL Type |
---|---|---|
(key).street1 | (key)_street1 | text |
(key).street2 | (key)_street2 | text |
(key).name | (key)_building_name | text |
(key).shop | (key)_shop | text |
(key).number | (key)_number | text |
(key).state | (key)_state | text |
(key).postcode | (key)_postcode | text |
(key).suburb | (key)_suburb | text |
(key).geo | (key)_geo | double precision array |
s3file
TypeKeystone Field | Column Name | PostgreSQL Type |
---|---|---|
(key).filename | (key)_filename | text |
(key).originalname | (key)_originalname | text |
(key).path | (key)_path | text |
(key).size | (key)_size | int |
(key).filetype | (key)_filetype | text |
(key).url | (key)_url | text |
cloudinaryimage
TypeKeystone Field | Column Name | PostgreSQL Type |
---|---|---|
(key).public_id | (key)_public_id | text |
(key).version | (key)_version | int |
(key).signature | (key)_signature | text |
(key).width | (key)_width | int |
(key).height | (key)_height | int |
(key).format | (key)_format | text |
(key).resource_type | (key)_resource_type | text |
(key).url | (key)_url | text |
(key).secure_url | (key)_secure_url | text |
Column names are forced to lowercase and restricted to the character set [a-zA-Z0-9_]
.
Any whitespace, hyphens or stops are replaced with underscores.
(TODO: Add configuration options for this)
Developed by John Molomby at Thinkmill for Keystone.js.
FAQs
A basic generator of MoSQL YAML config files for KeystoneJS models
The npm package @thinkmill/keystone-mosql-yaml-gen receives a total of 4 weekly downloads. As such, @thinkmill/keystone-mosql-yaml-gen popularity was classified as not popular.
We found that @thinkmill/keystone-mosql-yaml-gen demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 16 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
vlt's new "reproduce" tool verifies npm packages against their source code, outperforming traditional provenance adoption in the JavaScript ecosystem.
Research
Security News
Socket researchers uncovered a malicious PyPI package exploiting Deezer’s API to enable coordinated music piracy through API abuse and C2 server control.
Research
The Socket Research Team discovered a malicious npm package, '@ton-wallet/create', stealing cryptocurrency wallet keys from developers and users in the TON ecosystem.