iobroker.parser
Advanced tools
This adapter uses Sentry libraries to automatically report exceptions and code errors to the developers. For more details and for information how to disable the error reporting see Sentry-Plugin Documentation! Sentry reporting is used starting with js-controller 3.0.
This adapter parses data received via URL or from a file, by using regular expressions. For each rule being configured in the settings of this adapter, a state will be created under parser.<instance number> and filled and updated with the parsed information.
This default poll interval value will be used if no individual poll interval value is specified for an entry in the configuration table (column: "Interval"). The interval is in milliseconds and defines how often the link or file is being read and the states are being updated.
Note: Do not use a too aggressive poll interval especially for website URLs. For example, if you want to retrieve the price of your shares from a certain website, you probably should be good with an interval of just every 24 hours (= 86400000 ms), if you are not a day trader. If you try to retrieve data from certain URLs too often, the website may ban you and put you on a server blacklist. So please use the poll interval with care.
Specify how long the adapter waits for an HTTP response when doing website queries
Specify how long the adapter waits between making HTTP requests when performing remote queries. Useful when retrieving data from slow hosts or over slow connections to avoid overloading either one. Zero (default) means no delay.
This delay is on a per-host basis. If remote queries are configured to fetch from multiple remote hosts, each host will be queried in parallel.
The delay is a minimum value between initiating each request. I.e., if a query takes longer than this delay parameter to be read, the next will start instantly the read completes.
Specify if self-signed/invalid SSL/TLS certificates are accepted or declined when doing HTTPS requests
Specify to use an insecure HTTP parser that accepts invalid HTTP headers. This may allow interoperability with non-conformance HTTP implementations. Using the insecure parser should be avoided.
Click the "Plus" button to add a new entry to the table.
Performance Note: If you enter the same URL or filename more than once into different table rows, and if the values of the column "Interval" are the same, the content of the URL or filename will be retrieved only once and cached for processing multiple table rows matching URL/filename and Interval. This allows you to apply multiple regex (so multiple table rows) to a single URL or filename, without the need of retrieving the data multiple times from the source.
Table fields:
parser.<instance number>. Spaces are not allowed. You can use dots . as separator to create sub folders. Example: Shares.Microsoft.Current will result in parser.<instance number>.Shares.Microsoft.Current.https://darksky.net/forecast/48.1371,11.5754/si24/de (weather information Munich), or /opt/iobroker/test/testdata.txt (file from within ioBroker).temp swip">(-?\d+)˚< for the line above.°C, €, GB, etc.| Name | URL or file name | RegEx | Role | Type | Unit | Interval |
|---|---|---|---|---|---|---|
| temperatureMunich | https://darksky.net/forecast/48.1371,11.5754/si24/de | temp swip">(-?\d+)˚< | temperature | number | °C | 180000 |
| forumRunning | http://forum.iobroker.net/ | Forum | indicator | boolean | 60000 | |
| cloudRunning | https://iobroker.net/ | Privacy Notice | indicator | boolean | 60000 | |
| cpuTemperature | /sys/devices/virtual/thermal/thermal_zone0/temp | (.*) | temperature | number | °C | 30000 |
| stockPrice.Visa | https://www.finanzen.net/aktien/visa-aktie | \d{0,3},\d{2}(?=<span>EUR<\/span>) | value | number | € | 86400000 |
| kleinanzeigen | https://www.ebay-kleinanzeigen.de/s-iobroker/k0 | data-href="(.*?)."> | default | string | 600000 |
Note: While applying regex to the retrieved URL/file data, all line breaks will be replaced with spaces to allow multi-line search.
Regular expressions are a powerful tool to parse and extract certain data from strings, and even more important: it allows to extract certain values/text from a given string (like from the HTML of a webpage, or text from a file) by applying rules.
For boolean types, the regex is rather simple. For numeric types, you should mark the number with brackets - (). E.g. to extract the number from The temperature is 5°C you should use (\d+) expression.
Further information on RegExp:
.at matches any three-character string ending with at, including hat, cat, and bat.[hc]at matches hat and cat.[^b]at matches all strings matched by .at except bat.[^hc]at matches all strings matched by .at other than hat and cat.^[hc]at matches hat and cat, but only at the beginning of the string or line.[hc]at$ matches hat and cat, but only at the end of the string or line.\[.\] matches any single character surrounded by [ and ] since the brackets are escaped, for example: [a] and [b].s.\* matches s followed by zero or more characters, for example: s and saw and seed.[hc]+at matches hat, cat, hhat, chat, hcat, cchchat, and so on, but not at.[hc]?at matches hat, cat, and at.[hc]\*at matches hat, cat, hhat, chat, hcat, cchchat, at, and so on.cat|dog matches cat or dog.(\d+) - get the number from stringnow (\w+) later - get the word between now and later(-?\d+) get number (both negative and positive numbers)[+-]?([0-9]+.?[0-9]|.[0-9]+) get a number with decimal places (and . as decimal separator)[+-]?([0-9]+,?[0-9]|,[0-9]+) get a number with decimal places (and , as decimal separator)on("parser.0.kleinanzeigen", (obj) => {
sendTo("telegram.0", {
text: "https://www.ebay-kleinanzeigen.de" + obj.state.val,
});
});
Values can have quality codes:
Additionally, to the polling interval, the parsing of specific rules can be triggered by writing of empty value (false, 0, '' - depends on the type of state) to the state with false acknowledge flag.
In this case, the value will be read from the URL/file and parsed immediately.
You can also send a message to adapter with sendTo command:
sendTo("parser.0", "trigger", "temperatureMunich" /* name of rule, or parser.0.temperatureMunich */, result => {
console.log(JSON.stringify(result)); // {"value": 10, "error": null}
});
The MIT License (MIT)
Copyright (c) 2017-2023 bluefox dogafox@gmail.com
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
FAQs
Parses web pages and files for values.
The npm package iobroker.parser receives a total of 19 weekly downloads. As such, iobroker.parser popularity was classified as not popular.
We found that iobroker.parser demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 0 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
A malicious npm package disguised as a WhatsApp client is exploiting authentication flows with a remote kill switch to exfiltrate data and destroy files.
Security News
PyPI now supports digital attestations, enhancing security and trust by allowing package maintainers to verify the authenticity of Python packages.
Security News
GitHub removed 27 malicious pull requests attempting to inject harmful code across multiple open source repositories, in another round of low-effort attacks.