Research
Security News
Quasar RAT Disguised as an npm Package for Detecting Vulnerabilities in Ethereum Smart Contracts
Socket researchers uncover a malicious npm package posing as a tool for detecting vulnerabilities in Etherium smart contracts.
@ably/delta-codec
Advanced tools
Utility library supporting those consuming delta streams from Ably without using the official JavaScript client library (e.g. MQTT and SSE applications).
A Vcdiff codec library supporting the Vcdiff delta format, as defined by RFC 3284. This library supports developers who need to consume delta streams from Ably without using the official JavaScript client library (e.g. for MQTT and SSE applications).
Throughout this documentation, and within the code itself, we refer to a Vcdiff payload as a 'delta'. Elsewhere such delta payloads may be referred to as patches or diffs, but for consistency within this repository we stick to the terms 'delta' and 'deltas'.
npm install @ably/delta-codec
and require as:
var deltaCodec = require('@ably/delta-codec');
The VcdiffDecoder
constructor provides the most basic entry point to the public API. It provides a stateful way of applying a stream of Vcdiff deltas, producing a new value after each delta has been applied to the previous value.
First provide the base value, upon which the first delta will be applied using the 'setBase' method:
let decoder = new deltaCodec.VcdiffDecoder();
decoder.setBase(value);
Once the decoder has been initialized like this, then each subsequent delta is applied using the applyDelta
method:
let result = decoder.applyDelta(delta);
// TODO call method on result to get the value format you require
The decoder
retains the current value, with the result of this method encapsulating that new value now that delta
has been applied to the previous value.
The result
of this method offers methods to allow you to access the new current value in the format you require:
asUint8Array()
: a Uint8Array
, being the new current value, as received (i.e. 'raw' bytes)asUtf8String()
: a string
, decoded from data using UTF-8asObject()
: a JavaScript object, decoded from data using JSON.parse
setBase(value)
Instance MethodAccepts a single value
argument, the 'base', which may be ArrayBuffer
, Uint8Array
, Buffer
or string
.
If a string
is supplied then it will be UTF-8 encoded by the library before being stored as the current value.
This is freeform data, as specified by the application.
The decoder also exposes an alternative method, setBase64Base(value)
, where the single value
argument must be string
and is Base64 decoded by the library before being stored as the current value.
applyDelta(delta)
Instance MethodAccepts a single delta
argument which may be ArrayBuffer
, Uint8Array
or Buffer
.
This is a Vcdiff format delta.
The decoder also exposes an alternative method, applyBase64Delta(delta)
, where the single delta
argument must be string
and is Base64 decoded by the library before being applied as a Vcdiff format delta to the current value.
isDelta(data)
Static MethodAccepts a single data
argument which may be ArrayBuffer
, Uint8Array
or Buffer
.
Returns true
only if data
has a Vcdiff delta header.
This method can be used on receipt of a binary payload to detect whether it should be interpreted as an absolute value or as a delta to be applied to the previous value. Such 'sniffing' should be avoided where there is metadata available alongside received payloads to indicate whether they are deltas or not, as is the case when receiving a stream of enveloped data from Ably over SSE (see example).
The CheckedVcdiffDecoder
is a variant of VcdiffDecoder
that can be used when the values and the deltas applied to them have unique identifiers. The 'set' and 'apply' methods on the checked decoder have the same names but require additional arguments for these identifiers:
applyDelta(delta, deltaId, baseId)
applyBase64Delta(delta, deltaId, baseId)
setBase(value, baseId)
setBase64Base(value, baseId)
The baseId
argument supplied to the 'set' methods and the deltaId
arguments supplied to the 'apply' methods are stored alongside the current value and then compared to the baseId
argument supplied in subsequent 'apply' calls.
An Error
is thrown if there is a mismatch.
By default the event data received from Ably is enveloped in JSON format. We decode this data and inspect the Ably formatted message contents in order to establish whether the data in this message is an absolute value or a delta to be applied to the previously received value.
const deltaCodec = require('@ably/delta-codec');
const EventSource = require('eventsource');
const prefix = '[?delta=vcdiff]';
const url = `https://realtime.ably.io/event-stream?channels=${prefix}${CHANNEL_NAME}&v=1.2&key=${APP_KEY}`;
const eventSource = new EventSource(url);
const decoder = new deltaCodec.CheckedVcdiffDecoder();
eventSource.onmessage = function onEventSourceMessage(event) {
const message = JSON.parse(event.data);
let value;
const deltaExtras = (message.extras && message.extras.delta) ? message.extras.delta : null;
if (deltaExtras) {
if (deltaExtras.format !== 'vcdiff') {
throw new Error(`Delta format ${deltaExtras.format} not understood.`);
}
value = decoder.applyBase64Delta(message.data, message.id, deltaExtras.from).asUtf8String();
} else {
value = message.data;
decoder.setBase(value, message.id);
}
console.log(`received: ${value}`);
};
eventSource.onerror = function onEventSourceError(event) {
console.log(`error: ${event.data}`);
};
For this example we have subscribed to Ably as our event source and specified that we do not want the inbound event data to be JSON enveloped. Without envelopes the events will be smaller, taking up less transmission bandwidth, however this then means we need 'sniff' each inbound event's data to identify whether it is an absolute value or a delta to be applied to the previously received value.
Absolute values are sent to us as strings, ready to use. Deltas are sent to us as Base64 encoded binary.
const deltaCodec = require('@ably/delta-codec');
const EventSource = require('eventsource');
const prefix = '[?delta=vcdiff]';
const url = `https://realtime.ably.io/event-stream?channels=${prefix}${CHANNEL_NAME}&v=1.2&key=${APP_KEY}&enveloped=false`;
const eventSource = new EventSource(url);
const decoder = new deltaCodec.VcdiffDecoder();
eventSource.onmessage = function onEventSourceMessage(event) {
const stringData = event.data;
let value;
if (deltaCodec.VcdiffDecoder.isBase64Delta(stringData)) {
value = decoder.applyBase64Delta(stringData).asUtf8String();
} else {
value = stringData;
decoder.setBase(value);
}
console.log(`received: ${value}`);
};
eventSource.onerror = function onEventSourceError(event) {
console.log(`error: ${event.data}`);
};
The raw binary data received over MQTT has no encoding or other form of envelope encapsulating it. We need to 'sniff' each inbound payload to identify whether it is an absolute value or a delta to be applied to the previously received value. In this example we are transporting UTF-8 encoded strings.
const deltaCodec = require('@ably/delta-codec');
const mqtt = require('mqtt');
const brokerUrl = `mqtts://mqtt.ably.io`;
const options = {
username: APP_KEY_NAME,
password: APP_KEY_SECRET,
};
const prefix = '[?delta=vcdiff]';
const client = mqtt.connect(brokerUrl, options);
client.on('connect', () => {
client.subscribe(`${prefix}${CHANNEL_NAME}`);
});
const decoder = new deltaCodec.VcdiffDecoder();
client.on('message', (topic, message) => {
let value;
if (deltaCodec.VcdiffDecoder.isDelta(message)) {
value = decoder.applyDelta(message).asUtf8String();
} else {
decoder.setBase(message);
value = message.toString();
}
console.log(`received: ${value}`);
});
You can trigger a build using Webpack with:
npm run grunt -- build
which creates delta-codec.js
and delta-codec.min.js
in the dist
folder.
To run tests in all runtimes (Node and browsers):
npm test
To run tests on a single runtime:
npm run grunt -- test:node
npm run grunt -- test:browser:local
npm run grunt -- test:browser:remote
Known Issue:
When testing in a local browser either using npm run grunt -- test:browser:local
or indirectly by using npm test
on macOS, you may see a "segmentation fault". Launch the Console app to find the associated crash reports, of which there will be too. More information in issue #7.
Remote browser testing supported by
for which you will need to configure environment variables for BROWSERSTACK_USERNAME
and BROWSERSTACK_ACCESSKEY
.
On the master
branch:
/dist
folder by running npm run grunt -- release:patch
(or "major", "minor" or "prepatch" as appropriate - see grunt-bump Usage Examples)git push origin master --follow-tags
npm publish . --access public
(this package is configured to require that 2FA is used by publishers)npm run grunt -- publish-cdn
(operable by Ably staff only)FAQs
Utility library supporting those consuming delta streams from Ably without using the official JavaScript client library (e.g. MQTT and SSE applications).
We found that @ably/delta-codec demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket researchers uncover a malicious npm package posing as a tool for detecting vulnerabilities in Etherium smart contracts.
Security News
Research
A supply chain attack on Rspack's npm packages injected cryptomining malware, potentially impacting thousands of developers.
Research
Security News
Socket researchers discovered a malware campaign on npm delivering the Skuld infostealer via typosquatted packages, exposing sensitive data.