Security News
tea.xyz Spam Plagues npm and RubyGems Package Registries
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
@exodus/buffer-layout
Advanced tools
Readme
buffer-layout is a utility module implemented in pure JavaScript that supports translations between JavaScript values and Buffers. It is made available through github and released under the MIT license.
Layout support is provided for these types of data:
Development and testing is done using Node.js, supporting versions 4.5
and later. Install with npm install buffer-layout
.
All examples are from the test/examples.js
unit test and assume the
following context:
const assert = require('assert');
const util = require('util');
const lo = require('buffer-layout');
The examples give only a taste of what can be done. Structures, unions, and sequences can nest; union discriminators can be within the union or external to it; sequence and blob lengths may be fixed or read from the buffer.
For full details see the documentation.
The C definition:
int16_t arr[4] = { 1, -1, 3, -3 };
The buffer-layout way:
const ds = lo.seq(lo.s16(), 4);
const b = Buffer.alloc(8);
assert.equal(ds.encode([1, -1, 3, -3], b), 4 * 2);
assert.equal(Buffer.from('0100ffff0300fdff', 'hex').compare(b), 0);
assert.deepEqual(ds.decode(b), [1, -1, 3, -3]);
struct
on a 32-bit little-endian machineThe C definition:
struct ds {
uint8_t v;
uint32_t u32;
} st;
The buffer-layout way:
const ds = lo.struct([lo.u8('v'),
lo.seq(lo.u8(), 3), // alignment padding
lo.u32('u32')]);
assert.equal(ds.offsetOf('u32'), 4);
const b = Buffer.alloc(8);
b.fill(0xbd);
assert.equal(ds.encode({v: 1, u32: 0x12345678}, b), 1 + 3 + 4);
assert.equal(Buffer.from('01bdbdbd78563412', 'hex').compare(b), 0);
assert.deepEqual(ds.decode(b), {v: 1, u32: 0x12345678});
Note that the C language requires padding which must be explicitly added in the buffer-layout structure definition. Since the padding is not accessible, the corresponding layout has no property.
See Structure.
struct
on a 32-bit little-endian machineThe C definition:
struct ds {
uint8_t v;
uint32_t u32;
} __attribute__((__packed__)) st;
The buffer-layout way:
const ds = lo.struct([lo.u8('v'),
lo.u32('u32')]);
assert.equal(ds.offsetOf('u32'), 1);
const b = Buffer.alloc(5);
b.fill(0xbd);
assert.equal(ds.encode({v: 1, u32: 0x12345678}, b), 1 + 4);
assert.equal(Buffer.from('0178563412', 'hex').compare(b), 0);
assert.deepEqual(ds.decode(b), {v: 1, u32: 0x12345678});
Assume a 5-byte packed structure where the interpretation of the last four bytes depends on the first byte. The C definition:
struct {
uint8_t t;
union ds {
uint8_t u8[4]; // default interpretation
int16_t s16[2]; // when t is 'h'
uint32_t u32; // when t is 'w'
float f32; // when t is 'f'
} u;
} __attribute__((__packed__)) un;
The buffer-layout way:
const t = lo.u8('t');
const un = lo.union(t, lo.seq(lo.u8(), 4, 'u8'));
const nul = un.addVariant('n'.charCodeAt(0), 'nul');
const u32 = un.addVariant('w'.charCodeAt(0), lo.u32(), 'u32');
const s16 = un.addVariant('h'.charCodeAt(0), lo.seq(lo.s16(), 2), 's16');
const f32 = un.addVariant('f'.charCodeAt(0), lo.f32(), 'f32');
const b = Buffer.alloc(un.span);
assert.deepEqual(un.decode(b), {t: 0, u8: [0, 0, 0, 0]});
assert.deepEqual(un.decode(Buffer.from('6e01020304', 'hex')),
{nul: true});
assert.deepEqual(un.decode(Buffer.from('7778563412', 'hex')),
{u32: 0x12345678});
assert.deepEqual(un.decode(Buffer.from('660000bd41', 'hex')),
{f32: 23.625});
assert.deepEqual(un.decode(Buffer.from('a5a5a5a5a5', 'hex')),
{t: 0xa5, u8: [0xa5, 0xa5, 0xa5, 0xa5]});
assert.equal(s16.encode({s16: [123, -123]}, b), 1 + 2 * 2);
assert.equal(Buffer.from('687b0085ff', 'hex').compare(b), 0);
See Union.
The C definition:
struct ds {
unsigned int b00l03: 3;
unsigned int flg03: 1;
unsigned int b04l18: 24;
unsigned int b1Cl04: 4;
} st;
The buffer-layout way:
const ds = lo.bits(lo.u32());
const b = Buffer.alloc(4);
ds.addField(3, 'b00l03');
ds.addBoolean('flg03');
ds.addField(24, 'b04l18');
ds.addField(4, 'b1Cl04');
b.fill(0xff);
assert.equal(ds.encode({b00l03: 3, b04l18: 24, b1Cl04: 4}, b), 4);
assert.equal(Buffer.from('8b010040', 'hex').compare(b), 0);
assert.deepEqual(ds.decode(b),
{b00l03: 3, flg03: true, b04l18: 24, b1Cl04: 4});
See BitStructure.
The C definition:
uint64_t v = 0x0102030405060708ULL;
The buffer-layout way:
const ds = lo.nu64be();
const b = Buffer.from('0102030405060708', 'hex');
const v = 72623859790382856;
const nv = v - 6;
assert.equal(v, nv);
assert.equal(ds.decode(b), nv);
Note that because the exact value is not less than 2^53 it cannot be represented as a JavaScript Number, and is instead approximated by a nearby representable integer that is equivalent within Numbers.
See NearUInt64.
The C definition:
const char str[] = "hi!";
The buffer-layout way:
const ds = lo.cstr();
const b = Buffer.alloc(8);
assert.equal(ds.encode('hi!', b), 3 + 1);
const slen = ds.getSpan(b);
assert.equal(slen, 4);
assert.equal(Buffer.from('68692100', 'hex').compare(b.slice(0, slen)), 0);
assert.equal(ds.decode(b), 'hi!');
See CString.
The buffer-layout way:
const ds = lo.blob(4);
const b = Buffer.from('0102030405060708', 'hex');
assert.equal(Buffer.from('03040506', 'hex').compare(ds.decode(b, 2)), 0);
See Blob.
The buffer-layout way:
const pr = lo.seq(lo.cstr(), 2);
const n = lo.u8('n');
const vla = lo.seq(pr, lo.offset(n, -1), 'a');
const st = lo.struct([n, vla], 'st');
const b = Buffer.alloc(32);
const arr = [['k1', 'v1'], ['k2', 'v2'], ['k3', 'etc']];
b.fill(0);
assert.equal(st.encode({a: arr}, b),
1 + (2 * ((2 + 1) + (2 + 1)) + (2 + 1) + (3 + 1)));
const span = st.getSpan(b);
assert.equal(span, 20);
assert.equal(Buffer.from('036b31007631006b32007632006b330065746300', 'hex')
.compare(b.slice(0, span)), 0);
assert.deepEqual(st.decode(b), {n: 3, a: arr});
See OffsetLayout.
When data is obtained over a packetized interface the length of the packet can provide implicit limits on the last field.
The C definition:
struct ds {
uint8_t prop;
uint16_t data[];
};
The buffer-layout way:
const st = lo.struct([lo.u8('prop'),
lo.seq(lo.u16(),
lo.greedy(lo.u16().span),
'data')],
'ds');
const b = Buffer.from('21010002030405', 'hex');
assert.deepEqual(st.decode(b), {prop: 33, data: [0x0001, 0x0302, 0x0504]});
b.fill(0xFF);
assert.equal(st.encode({prop: 9, data: [5, 6]}, b), 1 + 2 * 2);
assert.equal(Buffer.from('0905000600FFFF', 'hex').compare(b), 0);
Storing arbitrary data using a leading byte to identify the content then a value that takes up only as much room as is necessary.
The example also shows how to extend the variant recognition API to support abitrary constant without consuming space for them in the encoded union. This could be used to make something similar to BSON.
Here's the code that defines the union, the variants, and the
recognition of true
and false
values for b
as distinct variants:
const un = lo.union(lo.u8('t'));
const u8 = un.addVariant('B'.charCodeAt(0), lo.u8(), 'u8');
const s16 = un.addVariant('h'.charCodeAt(0), lo.s16(), 's16');
const s48 = un.addVariant('Q'.charCodeAt(0), lo.s48(), 's48');
const cstr = un.addVariant('s'.charCodeAt(0), lo.cstr(), 'str');
const tr = un.addVariant('T'.charCodeAt(0), lo.const(true), 'b');
const fa = un.addVariant('F'.charCodeAt(0), lo.const(false), 'b');
const b = Buffer.alloc(1 + 6);
un.configGetSourceVariant(function(src) {
if (src.hasOwnProperty('b')) {
return src.b ? tr : fa;
}
return this.defaultGetSourceVariant(src);
});
And here are examples of encoding, checking the encoded length, and decoding each of the alternatives:
b.fill(0xff);
assert.equal(un.encode({u8: 1}, b), 1 + 1);
assert.equal(un.getSpan(b), 2);
assert.equal(Buffer.from('4201ffffffffff', 'hex').compare(b), 0);
assert.equal(un.decode(b).u8, 1);
b.fill(0xff);
assert.equal(un.encode({s16: -32000}, b), 1 + 2);
assert.equal(un.getSpan(b), 3);
assert.equal(Buffer.from('680083ffffffff', 'hex').compare(b), 0);
assert.equal(un.decode(b).s16, -32000);
b.fill(0xff);
const v48 = Math.pow(2, 47) - 1;
assert.equal(un.encode({s48: v48}, b), 1 + 6);
assert.equal(un.getSpan(b), 7);
assert.equal(Buffer.from('51ffffffffff7f', 'hex').compare(b), 0);
assert.equal(un.decode(b).s48, v48);
b.fill(0xff);
assert.equal(un.encode({b: true}, b), 1);
assert.equal(un.getSpan(b), 1);
assert.equal(Buffer.from('54ffffffffffff', 'hex').compare(b), 0);
assert.strictEqual(un.decode(b).b, true);
b.fill(0xff);
assert.equal(un.encode({b: false}, b), 1);
assert.equal(un.getSpan(b), 1);
assert.equal(Buffer.from('46ffffffffffff', 'hex').compare(b), 0);
assert.strictEqual(un.decode(b).b, false);
NOTE This code tickles a long-standing bug in Buffer.writeInt{L,B}E; if you are using Node prior to 4.2.4 or 5.2.0 you should update.
FAQs
Translation between JavaScript values and Buffers
The npm package @exodus/buffer-layout receives a total of 3,286 weekly downloads. As such, @exodus/buffer-layout popularity was classified as popular.
We found that @exodus/buffer-layout demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 84 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
Security News
As cyber threats become more autonomous, AI-powered defenses are crucial for businesses to stay ahead of attackers who can exploit software vulnerabilities at scale.
Security News
UnitedHealth Group disclosed that the ransomware attack on Change Healthcare compromised protected health information for millions in the U.S., with estimated costs to the company expected to reach $1 billion.