High speed Synchronous and Asynchronous access to M-like databases from Node.js.
High speed Synchronous and Asynchronous access to M-like databases from Node.js.
Chris Munt cmunt@mgateway.com
21 April 2020, M/Gateway Developments Ltd http://www.mgateway.com
mg-dbx is a Node.js addon written in C++. It is distributed as C++ source code and the NPM installation procedure will expect a C++ compiler to be present on the target system.
Linux systems can use the freely available GNU C++ compiler (g++) which can be installed as follows.
Ubuntu:
apt-get install g++
Red Hat and CentOS:
yum install gcc-c++
Apple OS X can use the freely available Xcode development environment.
There are two options for Windows, both of which are free:
If the Windows machine is not set up for systems development, building native Addon modules for this platform from C++ source can be quite arduous. There is some helpful advice available at:
Alternatively there are built Windows x64 binaries available from:
Assuming that Node.js is already installed and a C++ compiler is available to the installation process:
npm install mg-dbx
This command will create the mg-dbx addon (mg-dbx.node).
Most mg-dbx methods are capable of operating either synchronously or asynchronously. For an operation to complete asynchronously, simply supply a suitable callback as the last argument in the call.
The first step is to add mg-dbx to your Node.js script
var dbx = require('mg-dbx').dbx;
var db = new dbx();
In the following examples, modify all paths (and any user names and passwords) to match those of your own installation.
Assuming Cache is installed under /opt/cache20181/
var open = db.open({
type: "Cache",
path:"/opt/cache20181/mgr",
username: "_SYSTEM",
password: "SYS",
namespace: "USER"
});
Assuming IRIS is installed under /opt/IRIS20181/
var open = db.open({
type: "IRIS",
path:"/opt/IRIS20181/mgr",
username: "_SYSTEM",
password: "SYS",
namespace: "USER"
});
Assuming an 'out of the box' YottaDB installation under /usr/local/lib/yottadb/r122.
var envvars = "";
envvars = envvars + "ydb_dir=/root/.yottadb\n"
envvars = envvars + "ydb_rel=r1.22_x86_64\n"
envvars = envvars + "ydb_gbldir=/root/.yottadb/r1.22_x86_64/g/yottadb.gld\n"
envvars = envvars + "ydb_routines=/root/.yottadb/r1.22_x86_64/o*(/root/.yottadb/r1.22_x86_64/r /root/.yottadb/r) /usr/local/lib/yottadb/r122/libyottadbutil.so\n"
envvars = envvars + "ydb_ci=/usr/local/lib/yottadb/r122/cm.ci\n"
envvars = envvars + "\n"
var open = db.open({
type: "YottaDB",
path: "/usr/local/lib/yottadb/r122",
env_vars: envvars
});
current_namespace = db.namespace([<new_namespace>]);
Example 1 (Get the current Namespace):
var nspace = db.namespace();
Example 2 (Change the current Namespace):
var new_nspace = db.namespace("SAMPLES");
If the operation is successful this method will echo back the new Namespace name. If not successful, the method will return the name of the current (unchanged) Namespace.
global = db.mglobal(<global_name>[, <fixed_key>]);
Example (using a global named "Person"):
var person = db.mglobal("Person");
Synchronous:
var result = <global>.set(<key>, <data>);
Asynchronous:
<global>.set(<key>, <data>, callback(<error>, <result>));
Example:
person.set(1, "John Smith");
Synchronous:
var result = <global>.get(<key>);
Asynchronous:
<global>.get(<key>, callback(<error>, <result>));
Example:
var name = person.get(1);
Synchronous:
var result = <global>.delete(<key>);
Asynchronous:
<global>.delete(<key>, callback(<error>, <result>));
Example:
var name = person.delete(1);
Synchronous:
var result = <global>.defined(<key>);
Asynchronous:
<global>.defined(<key>, callback(<error>, <result>));
Example:
var name = person.defined(1);
Synchronous:
var result = <global>.next(<key>);
Asynchronous:
<global>.next(<key>, callback(<error>, <result>));
Example:
var key = "";
while ((key = person.next(key)) != "") {
console.log("\nPerson: " + key + ' : ' + person.get(key));
}
Synchronous:
var result = <global>.previous(<key>);
Asynchronous:
<global>.previous(<key>, callback(<error>, <result>));
Example:
var key = "";
while ((key = person.previous(key)) != "") {
console.log("\nPerson: " + key + ' : ' + person.get(key));
}
Synchronous:
var result = <global>.increment(<key>, <increment_value>);
Asynchronous:
<global>.increment(<key>, <increment_value>, callback(<error>, <result>));
Example (increment the value of the "counter" node by 1.5 and return the new value):
var result = person.increment("counter", 1.5);
Synchronous:
var result = <global>.lock(<key>, <timeout>);
Asynchronous:
<global>.lock(<key>, <timeout>, callback(<error>, <result>));
Example (lock global node '1' with a timeout of 30 seconds):
var result = person.lock(1, 30);
Synchronous:
var result = <global>.unlock(<key>);
Asynchronous:
<global>.unlock(<key>, callback(<error>, <result>));
Example (unlock global node '1'):
var result = person.unlock(1);
<global>.reset(<global_name>[, <fixed_key>]);
Example:
// Process orders for customer #1
customer_orders = db.mglobal("Customer", 1, "orders")
do_work ...
// Process orders for customer #2
customer_orders.reset("Customer", 2, "orders");
do_work ...
This facility provides high-performance techniques for traversing records held in database globals.
The first task is to specify the 'query' for the global traverse.
query = db.mglobalquery({global: <global_name>, key: [<seed_key>]}[, <options>]);
The 'options' object can contain the following properties:
multilevel: A boolean value (default: multilevel: false). Set to 'true' to return all descendant nodes from the specified 'seed_key'.
getdata: A boolean value (default: getdata: false). Set to 'true' to return any data values associated with each global node returned.
format: Format for output (default: not specified). If the output consists of multiple data elements, the return value (by default) is a JavaScript object made up of a 'key' array and an associated 'data' value. Set to "url" to return such data as a single URL escaped string including all key values ('key[1->n]') and any associated 'data' value.
Example (return all keys and names from the 'Person' global):
query = db.mglobalquery({global: "Person", key: [""]}, {multilevel: false, getdata: true});
In key order:
result = query.next();
In reverse key order:
result = query.previous();
In all cases these methods will return 'null' when the end of the dataset is reached.
Example 1 (return all key values from the 'Person' global - returns a simple variable):
query = db.mglobalquery({global: "Person", key: [""]});
while ((result = query.next()) !== null) {
console.log("result: " + result);
}
Example 2 (return all key values and names from the 'Person' global - returns an object):
query = db.mglobalquery({global: "Person", key: [""]}, multilevel: false, getdata: true);
while ((result = query.next()) !== null) {
console.log("result: " + JSON.stringify(result, null, '\t'));
}
Example 3 (return all key values and names from the 'Person' global - returns a string):
query = db.mglobalquery({global: "Person", key: [""]}, multilevel: false, getdata: true, format: "url"});
while ((result = query.next()) !== null) {
console.log("result: " + result);
}
Example 4 (return all key values and names from the 'Person' global, including any descendant nodes):
query = db.mglobalquery({global: "Person", key: [""]}, {{multilevel: true, getdata: true});
while ((result = query.next()) !== null) {
console.log("result: " + JSON.stringify(result, null, '\t'));
}
query = db.mglobalquery({global: <seed_global_name>}, {globaldirectory: true});
Example (return all global names held in the current directory)
query = db.mglobalquery({global: ""}, {globaldirectory: true});
while ((result = query.next()) !== null) {
console.log("result: " + result);
}
Synchronous:
result = db.function(<function>, <parameters>);
Asynchronous:
db.function(<function>, <parameters>, callback(<error>, <result>));
Example:
M routine called 'math':
add(a, b) ; Add two numbers together
quit (a+b)
JavaScript invocation:
result = db.function("add^math", 2, 3);
Synchronous:
result = db.classmethod(<class_name>, <classmethod_name>, <parameters>);
Asynchronous:
db.classmethod(<class_name>, <classmethod_name>, <parameters>, callback(<error>, <result>));
Example (Encode a date to internal storage format):
result = db.classmethod("%Library.Date", "DisplayToLogical", "10/10/2019");
The following simple class will be used to illustrate this facility.
Class User.Person Extends %Persistent
{
Property Number As %Integer;
Property Name As %String;
Property DateOfBirth As %Date;
Method Age(AtDate As %Integer) As %Integer
{
Quit (AtDate - ..DateOfBirth) \ 365.25
}
}
person = db.classmethod("User.Person", "%New");
Add Data:
result = person.setproperty("Number", 1);
result = person.setproperty("Name", "John Smith");
result = person.setproperty("DateOfBirth", "12/8/1995");
Save the object record:
result = person.method("%Save");
Retrieve data for object %Id of 1.
person = db.classmethod("User.Person", "%OpenId", 1);
Return properties:
var number = person.getproperty("Number");
var name = person.getproperty("Name");
var dob = person.getproperty("DateOfBirth");
Calculate person's age at a particular date:
today = db.classmethod("%Library.Date", "DisplayToLogical", "10/10/2019");
var age = person.method("Age", today);
Once created, it is possible to reuse containers holding previously instantiated objects using the reset() method. Using this technique helps to reduce memory usage in the Node.js environment.
Example 1 Reset a container to hold a new instance:
person.reset("User.Person", "%New");
Example 2 Reset a container to hold an existing instance (object %Id of 2):
person.reset("User.Person", "%OpenId", 2);
mg-dbx provides direct access to the Open Source MGSQL engine (https://github.com/chrisemunt/mgsql) and InterSystems SQL (IRIS and Cache).
In order to use this facility two M routines need to be installed (%zmgsi and %zmgsis). These can be found in the GitHub source code repository (https://github.com/chrisemunt/mg-dbx)
The instructions given here assume a standard 'out of the box' installation of YottaDB deployed in the following location:
/usr/local/lib/yottadb/r122
The primary default location for routines:
/root/.yottadb/r1.22_x86_64/r
Copy all the routines (i.e. all files with an 'm' extension) held in the GitHub /yottadb directory to:
/root/.yottadb/r1.22_x86_64/r
Change directory to the following location and start a YottaDB command shell:
cd /usr/local/lib/yottadb/r122
./ydb
Link all the zmgsi routines and check the installation:
do ylink^%zmgsi
do ^%zmgsi
M/Gateway Developments Ltd - Service Integration Gateway
Version: 3.2; Revision 3 (1 November 2019)
Note that the version of zmgsi is successfully displayed.
Finally, add the following lines to the interface file (cm.ci in the example used in the db.open() method).
sqlemg: ydb_char_t * sqlemg^%zmgsis(I:ydb_char_t*, I:ydb_char_t *, I:ydb_char_t *)
sqlrow: ydb_char_t * sqlrow^%zmgsis(I:ydb_char_t*, I:ydb_char_t *, I:ydb_char_t *)
sqldel: ydb_char_t * sqldel^%zmgsis(I:ydb_char_t*, I:ydb_char_t *)
Log in to the Manager UCI and, using the %RI utility (or similar) load the zmgsi routines held in /m/zmgsi.ro. Change to your development UCI and check the installation:
do ^%zmgsi
M/Gateway Developments Ltd - Service Integration Gateway
Version: 3.2; Revision 3 (1 November 2019)
The first task is to specify the SQL query.
query = db.sql({sql: <sql_statement>[, type: <sql_engine>]);
Example 1 (using MGSQL):
query = db.sql({sql: "select * from person"});
Example 2 (using InterSystems SQL):
query = db.sql({sql: "select * from SQLUser.person", type: "Cache"});
Synchronous:
var result = <query>.execute();
Asynchronous:
<query>.execute(callback(<error>, <result>));
The result of query execution is an object containing the return code and state and any associated error message. The familiar ODBC return and status codes are used.
Example 1 (successful execution):
{
"sqlcode": 0,
"sqlstate": "00000",
}
Example 2 (unsuccessful execution):
{
"sqlcode": -1,
"sqlstate": "HY000",
"error": "no such table 'person'"
}
In result-set order:
result = query.next();
In reverse result-set order:
result = query.previous();
In all cases these methods will return 'null' when the end of the dataset is reached.
Example:
while ((row = query.next()) !== null) {
console.log("row: " + JSON.stringify(result, null, '\t'));
}
The output for each iteration is a row of the generated SQL result-set. For example:
{
"number": 1,
"name": "John Smith",
}
For 'select' queries that generate a result-set it is good practice to invoke the 'cleanup' method at the end to delete the result-set held in the database.
Synchronous:
var result = <query>.cleanup();
Asynchronous:
<query>.cleanup(callback(<error>, <result>));
Synchronous:
<query>.reset({sql: <sql_statement>[, type: <sql_engine>]);
Asynchronous:
<query>.reset({sql: <sql_statement>[, type: <sql_engine>], callback(<error>, <result>));
var result = db.version();
Example:
console.log("\nmg-dbx Version: " + db.version());
db.close();
Copyright (c) 2018-2020 M/Gateway Developments Ltd,
Surrey UK.
All rights reserved.
http://www.mgateway.com
Email: cmunt@mgateway.com
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
FAQs
High speed Synchronous and Asynchronous access to InterSystems Cache/IRIS and YottaDB from Node.js.
The npm package mg-dbx receives a total of 11 weekly downloads. As such, mg-dbx popularity was classified as not popular.
We found that mg-dbx demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket’s threat research team has detected six malicious npm packages typosquatting popular libraries to insert SSH backdoors.
Security News
MITRE's 2024 CWE Top 25 highlights critical software vulnerabilities like XSS, SQL Injection, and CSRF, reflecting shifts due to a refined ranking methodology.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.