Security News
Research
Data Theft Repackaged: A Case Study in Malicious Wrapper Packages on npm
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
This is a javascript binary serialization implementation for the Energistics Transfer protocol (ETP). ETP is a proposed specification for streaming real-time data from oil field drilling and production facilities. It uses websockets for transport and Apache Avro for serialization. Note that there is no need for a libary to serialize the JSON versions of objects, since JSON.parse and JSON.stringify can be used directly with the websocket messages. However, the schema validation methods described below can also be used with JSON objects.
This node package contains:
Note that a previous version of this library exists called node-etp and has been deprecated.
npm install etp
Clone the node folder from bitbucket.
c:\energistics\src git clone https://bitbucket.org/energistics/node-etp.git
Install into your project directory c:\ralfdemo>npm install c:\energistics\src\node-etp
load the library
> var etp = require("etp");
the ETP library wraps the etp-avro package. From this we can get readers and writers, etc.
> var avro = etp.avro;
Create a new schema cache object. This is used by the library to read, write, and validate schemas. Note the use of etp.SchemaCache() vs. avro.SchemaCache(schemas). The etp version of the SchemaCache class automatically loads the correct schemas, and has utility functions to deal with ETP messages.
>var sc = new etp.SchemaCache();
The following finds the schema associated with protocol 1, message 1. You can use it on incoming messages after messages, after decoding the header.
> sc.find(1, 1);
{ type: 'record',
namespace: 'Energistics.Protocol.ChannelStreaming',
name: 'ChannelDescribe',
messageType: '1',
protocol: '1',
fields: [ { name: 'uris', type: [Object] } ],
fullName: 'Energistics.Protocol.ChannelStreaming.ChannelDescribe',
depends: [] }
>
The decode function on the reader allows you to pass any array of values to decode a single datum (which can also be a structured type, so long as it is registered in the schema cache).
> var reader = new avro.BinaryReader(sc);
> var msg = reader.decode("string", [22, 0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x57, 0x6f, 0x72, 0x6c, 0x64]);
> console.log(msg);
Hello World
However, assuming you have received a binary message from a websocket object. You can attach the binary message to the reader by using the builtin js type Uint8Array. You can then reader the message header first, and then lookup the appropriate schema for the message boday.
reader = new avro.BinaryReader(sc, new Uint8Array(message.binaryData)),
header = reader.readDatum("Energistics.Datatypes.MessageHeader"),
body = reader.readDatum(this.schemaCache.find(header.protocol, header.messageType));
The writing functions work more or less the same way. The getBuffer() method gets the value which can be passed directly to the websocket send() method.
writer = new avro.BinaryWriter(sc);
writer.writeDatum("Energistics.Datatypes.MessageHeader", header);
writer.writeDatum(schemaName, message);
dataToSend = encoder.getBuffer();
2^53 is the largest effective integer value that we can put in a javascript Number type (which is a 64 bit IEEE double precision number) without loss of precision. Negative is 1 power of 2 smaller. After this point, avro longs will lose precision.
[128, 128, 128, 128, 128, 128, 128, 32] ==> 9007199254740992
[255, 255, 255, 255, 255, 255, 255, 15] ==> -4503599627370496
The schema cache is itself an associative array of schemas
> sc['Energistics.Datatypes.Version']
{ type: 'record',
namespace: 'Energistics.Datatypes',
name: 'Version',
fields:
[ { name: 'major', type: 'int' },
{ name: 'minor', type: 'int' },
{ name: 'revision', type: 'int' },
{ name: 'patch', type: 'int' } ],
fullName: 'Energistics.Datatypes.Version',
depends: [] }
>
The schema cache object can be used to validate JS objects. Validate does not return a value, simply throws if the input object does not match the schema. In testing and debugging phases, especially when you implement a new version of the protocol, validation is recommended for all messages.
> sc.validate('Energistics.Datatypes.Version', {major: 11, minor: 2, revision:3, patch:4});
undefined
> sc.validate('Energistics.Datatypes.Version', {major: 11, minor: 2, revision:3, patch:"string"});
TypeAssertionError: expected integer
at D:\test\etp\node_modules\etp\node_modules\etp-avro\lib\assert-type.js:212:13
at SchemaCache.validate (D:\test\etp\node_modules\etp\node_modules\etp-avro\lib\etp-avro.js:586:42)
at SchemaCache.validate (D:\test\etp\node_modules\etp\node_modules\etp-avro\lib\etp-avro.js:599:26)
at SchemaCache.validate (D:\test\etp\node_modules\etp\node_modules\etp-avro\lib\etp-avro.js:648:22)
at repl:1:4
at REPLServer.self.eval (repl.js:110:21)
at repl.js:249:20
at REPLServer.self.eval (repl.js:122:7)
at Interface.<anonymous> (repl.js:239:12)
at Interface.emit (events.js:95:17)
As noted above, you can use any old javascript object with the serializers and validation routines, but the library also provides concrete classs, within namespaces. The concrete classes initialize a hash with reasonable defaults for most fields, including any Avro default that is specified. They also keep a copy of the schema itself as the _schema member of the hash.
> var version = new etp.Messages.Energistics.Datatypes.Version();
undefined
> version
{ major: 0,
minor: 0,
revision: 0,
patch: 0,
_schema:
{ type: 'record',
namespace: 'Energistics.Datatypes',
name: 'Version',
fields:
[ [Object],
[Object],
[Object],
[Object] ],
fullName: 'Energistics.Datatypes.Version',
depends: [] } }
>
To use in a web browser application, simply copy either the webified or webified/minified version of the .js file from the node_modules/etp/lib/browser to an appropriate website folder and include this file a script tag.
<script type="text/javascript" src="js/etp.min.js"></script>
You can get it from the jsdelivr CDN as use in the code below.
Loading the browser modules creates a property on the window object with a full export of the library classes.
#!html
<html>
<head>
<meta charset="utf-8">
<title>ETP Test</title>
</head>
<body style="font-family: verdana; font-size: 9pt; color: black">
<div id="etp"></div>
<script src="http://cdn.jsdelivr.net/etp/latest/etp.min.js"></script>
<script>
function setup() {
var etp = window.etp;
// List all schema types
for (var i=0; i<etp.Schemas.types.length; i++) {
document.getElementById("console").innerHTML+= ( "<br/>" + etp.Schemas.types[i].fullName );
}
// de-serialize a binary avro array
var avro = etp.avro;
var schemaCache = new avro.SchemaCache( [] );
var reader = new avro.BinaryReader(schemaCache);
var msg = reader.decode("string", [22, 0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x57, 0x6f, 0x72, 0x6c, 0x64]);
document.getElementById("console").innerHTML+= ( "<br/><br/>Binary decode: " + msg );
}
window.onload = setup;
</script>
Hello ETP
<div id="console" />
</body>
</html>
FAQs
Javascript bindings for Energistics Transport Protocol (ETP)
The npm package etp receives a total of 47 weekly downloads. As such, etp popularity was classified as not popular.
We found that etp demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.