iobroker.sql
Advanced tools
This adapter saves state history into SQL DB.
Supports PostgreSQL, mysql, Microsoft SQL Server and sqlite. You can leave port 0 if default port is desired.
This adapter uses Sentry libraries to automatically report exceptions and code errors to the developers. For more details and for information how to disable the error reporting see Sentry-Plugin Documentation! Sentry reporting is used starting with js-controller 3.0.
Use localhost\instance for the host and check TCP/IP connections enabled.
https://msdn.microsoft.com/en-us/library/bb909712(v=vs.90).aspx
is "file"-DB and cannot manage too many events. If you have a big amount of data use the real DB, like PostgreSQL and co.
SQLite DB must not be installed extra. It is just a file on disk, but to install it you require build tools on your system. For linux, just write:
sudo apt-get install build-essential
For windows:
c:\>npm install --global --production windows-build-tools
and then reinstall the adapter, e.g:
cd /opt/iobroker
iobroker stop sql
npm install iobroker.sql --production
iobroker start sql
You can install mysql on linux systems as following:
apt-get install mysql-server mysql-client
mysql -uroot -p
CREATE USER 'iobroker'@'%' IDENTIFIED BY 'iobroker';
GRANT ALL PRIVILEGES ON * . * TO 'iobroker'@'%';
FLUSH PRIVILEGES;
If required edit /etc/mysql/my.cnf to set bind to IP-Address for remote connect.
Warning: iobroker user is "admin". If required give limited rights to iobroker user.
On the "windows" it can be easily installed via installer: https://dev.mysql.com/downloads/installer/.
Pay attention for authentication method. The new encryption algorithm in MySQL 8.0 is not yet supported by node.js and you must select legacy authentication method.
Default Database name is "iobroker", but it can be changed in the configuration.
This table is a list of adapter's instances, that wrote the entries. (state.from)
| DB | Name in query |
|---|---|
| MS-SQL | iobroker.dbo.sources |
| MySQL | iobroker.sources |
| PostgreSQL | sources |
| SQLite | sources |
Structure:
| Field | Type | Description |
|---|---|---|
| id | INTEGER NOT NULL PRIMARY KEY IDENTITY(1,1) | unique ID |
| name | varchar(255) / TEXT | instance of adapter, that wrote the entry |
Note: MS-SQL uses varchar(255), and others use TEXT
This table is a list of data points. (IDs)
| DB | Name in query |
|---|---|
| MS-SQL | iobroker.dbo.datapoints |
| MySQL | iobroker.datapoints |
| PostgreSQL | datapoints |
| SQLite | datapoints |
Structure:
| Field | Type | Description |
|---|---|---|
| id | INTEGER NOT NULL PRIMARY KEY IDENTITY(1,1) | unique ID |
| name | varchar(255) / TEXT | ID of variable, e.g. hm-rpc.0.JEQ283747.1.STATE |
| type | INTEGER | 0 - number, 1 - string, 2 - boolean |
Note: MS-SQL uses varchar(255), and others use TEXT
Values for states with type "number". ts means "time series".
| DB | Name in query |
|---|---|
| MS-SQL | iobroker.dbo.ts_number |
| MySQL | iobroker.ts_number |
| PostgreSQL | ts_number |
| SQLite | ts_number |
Structure:
| Field | Type | Description |
|---|---|---|
| id | INTEGER | ID of state from "Data points" table |
| ts | BIGINT / INTEGER | Time in ms till epoch. Can be converted to time with "new Date(ts)" |
| val | REAL | Value |
| ack | BIT/BOOLEAN | Is acknowledged: 0 - not ack, 1 - ack |
| _from | INTEGER | ID of source from "Sources" table |
| q | INTEGER | Quality as number. You can find description here |
Note: MS-SQL uses BIT, and others use BOOLEAN. SQLite uses for ts INTEGER and all others BIGINT.
The user can define additional to type number the functionality of "counters". For this purpose following table is created:
| DB | Name in the query |
|---|---|
| MS-SQL | iobroker.dbo.ts_counter |
| MySQL | iobroker.ts_counter |
| PostgreSQL | ts_counter |
| SQLite | ts_counter |
Structure:
| Field | Type | Description |
|---|---|---|
| id | INTEGER | ID of state from "Data points" table |
| ts | BIGINT / INTEGER | Time in ms till epoch. Can be converted to time with "new Date(ts)" |
| val | REAL | Value |
This table stores the values when the counter was exchanged and the value does not increased, but failed to zero or lower value.
Values for states with type "string".
| DB | Name in query |
|---|---|
| MS-SQL | iobroker.dbo.ts_string |
| MySQL | iobroker.ts_string |
| PostgreSQL | ts_string |
| SQLite | ts_string |
Structure:
| Field | Type | Description |
|---|---|---|
| id | INTEGER | ID of state from "Data points" table |
| ts | BIGINT | Time in ms till epoch. Can be converted to time with "new Date(ts)" |
| val | TEXT | Value |
| ack | BIT/BOOLEAN | Is acknowledged: 0 - not ack, 1 - ack |
| _from | INTEGER | ID of source from "Sources" table |
| q | INTEGER | Quality as number. You can find description here |
Note: MS-SQL uses BIT, and others use BOOLEAN. SQLite uses for ts INTEGER and all others BIGINT.
Values for states with type "boolean".
| DB | Name in query |
|---|---|
| MS-SQL | iobroker.dbo.ts_bool |
| MySQL | iobroker.ts_bool |
| PostgreSQL | ts_bool |
| SQLite | ts_bool |
Structure:
| Field | Type | Description |
|---|---|---|
| id | INTEGER | ID of state from "Data points" table |
| ts | BIGINT | Time in ms till epoch. Can be converted to time with "new Date(ts)" |
| val | BIT/BOOLEAN | Value |
| ack | BIT/BOOLEAN | Is acknowledged: 0 - not ack, 1 - ack |
| _from | INTEGER | ID of source from "Sources" table |
| q | INTEGER | Quality as number. You can find description here |
Note: MS-SQL uses BIT, and others use BOOLEAN. SQLite uses for ts INTEGER and all others BIGINT.
The user can execute custom queries on tables from javascript adapter:
sendTo('sql.0', 'query', 'SELECT * FROM datapoints', function (result) {
if (result.error) {
console.error(result.error);
} else {
// show result
console.log('Rows: ' + JSON.stringify(result.result));
}
});
Or get entries for the last hour for ID=system.adapter.admin.0.memRss
sendTo('sql.0', 'query', 'SELECT id FROM datapoints WHERE name="system.adapter.admin.0.memRss"', function (result) {
if (result.error) {
console.error(result.error);
} else {
// show result
console.log('Rows: ' + JSON.stringify(result.result));
var now = new Date();
now.setHours(-1);
sendTo('sql.0', 'query', 'SELECT * FROM ts_number WHERE ts >= ' + now.getTime() + ' AND id=' + result.result[0].id, function (result) {
console.log('Rows: ' + JSON.stringify(result.result));
});
}
});
If you want to write other data into the InfluxDB/SQL you can use the build in system function storeState. This function can also be used to convert data from other History adapters like History or SQL.
The given IDs are not checked against the ioBroker database and do not need to be set up there, but can only be accessed directly.
The Message can have one of the following three formats:
{id: 'adapter.0.device.counter', state: {val: 1, ts: 10239499}}{id: 'adapter.0.device.counter', state: [{val: 1, ts: 10239499}, {val: 2, ts: 10239599}, {val: 3, ts: 10239699}]}[{id: 'adapter.0.device.counter1', state: {val: 1, ts: 10239499}, {id: 'adapter.0.device.counter2', state: {val: 2, ts: 10239599}]Additionally, you can add attribute rules: true to activate all rules, like counter, changesOnly, de-bounce and so on: {id: 'adapter.0.device.counter', rules: true, state: [{val: 1, ts: 10239499}, {val: 2, ts: 10239599}, {val: 3, ts: 10239699}]}
If you want to delete entry from the Database you can use the build in system function delete:
sendTo('sql.0', 'delete', [
{id: 'mbus.0.counter.xxx, state: {ts: 1589458809352},
{id: 'mbus.0.counter.yyy, state: {ts: 1589458809353}
], result => console.log('deleted'));
To delete ALL history data for some data point execute:
sendTo('sql.0', 'deleteAll', [
{id: 'mbus.0.counter.xxx}
{id: 'mbus.0.counter.yyy}
], result => console.log('deleted'));
To delete history data for some data point and for some range execute:
sendTo('sql.0', 'deleteRange', [
{id: 'mbus.0.counter.xxx, start: '2019-01-01T00:00:00.000Z', end: '2019-12-31T23:59:59.999'},
{id: 'mbus.0.counter.yyy, start: 1589458809352, end: 1589458809353}
], result => console.log('deleted'));
Time could be ms since epoch or ans string, that could be converted by javascript Date object.
Values will be deleted including defined limits. ts >= start AND ts <= end
If you want to change entry's value, quality or acknowledge flag in the database you can use the build in system function update:
sendTo('sql.0', 'update', [
{id: 'mbus.0.counter.xxx, state: {ts: 1589458809352, val: 15, ack: true, q: 0},
{id: 'mbus.0.counter.xxx, state: {ts: 1589458809353, val: 16, ack: true, q: 0}
], result => console.log('deleted'));
ts is mandatory. At least one other flags must be included in state object.
Be careful with counters. The counters in DB will not be reset and you must handle it yourself.
Additional to custom queries, you can use build in system function getHistory:
var end = Date.now();
sendTo('sql.0', 'getHistory', {
id: 'system.adapter.admin.0.memRss',
options: {
start: end - 3600000,
end: end,
aggregate: 'minmax' // or 'none' to get raw values
}
}, function (result) {
for (var i = 0; i < result.result.length; i++) {
console.log(result.result[i].id + ' ' + new Date(result.result[i].ts).toISOString());
}
});
User can ask the value of some counter (type=number, counter=true) for specific period.
var now = Date.now();
// get consumption value for last 30 days
sendTo('sql.0', 'getCounter', {
id: 'system.adapter.admin.0.memRss',
options: {
start: now - 3600000 * 24 * 30,
end: now,
}
}, result => {
console.log(`In last 30 days the consumption was ${result.result} kWh`);
});
If the counter will be replaced it will be calculated too.
The adapter supports enabling and disabling of history logging via JavaScript and also retrieving the list of enabled data points with their settings.
The message requires to have the "id" of the data point. Additionally optional "options" to define the data point specific settings:
sendTo('sql.0', 'enableHistory', {
id: 'system.adapter.sql.0.memRss',
options: {
changesOnly: true,
debounce: 0,
retention: 31536000,
maxLength: 3,
changesMinDelta: 0.5,
aliasId: ''
}
}, function (result) {
if (result.error) {
console.log(result.error);
}
if (result.success) {
//successfull enabled
}
});
The message requires to have the "id" of the data point.
sendTo('sql.0', 'disableHistory', {
id: 'system.adapter.sql.0.memRss',
}, function (result) {
if (result.error) {
console.log(result.error);
}
if (result.success) {
//successfull enabled
}
});
The message has no parameters.
sendTo('sql.0', 'getEnabledDPs', {}, function (result) {
//result is object like:
{
"system.adapter.sql.0.memRss": {
"changesOnly":true,
"debounce":0,
"retention":31536000,
"maxLength":3,
"changesMinDelta":0.5,
"enabled":true,
"changesRelogInterval":0,
"aliasId": ""
}
...
}
});
BREAKING This version only accepts Node.js 10.x+ (because sqlite3 was upgraded)
The MIT License (MIT)
Copyright (c) 2015-2020 bluefox dogafox@gmail.com, Apollon77
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
FAQs
Log states into SQL database
The npm package iobroker.sql receives a total of 391 weekly downloads. As such, iobroker.sql popularity was classified as not popular.
We found that iobroker.sql demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 6 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
ENISA’s 2024 report highlights the EU’s top cybersecurity threats, including rising DDoS attacks, ransomware, supply chain vulnerabilities, and weaponized AI.
Security News
NIST's new password guidelines remove periodic changes and special character requirements, focusing on longer, more secure passwords for better authentication practices.
Security News
A record 2,709 developers participated in the 2024 Ruby on Rails Community Survey, revealing key tools, practices, and trends shaping the Rails ecosystem.