@bnaya/objectbuffer

ObjectBuffer: object-like API, backed by a [shared]arraybuffer 👀

For Modern browsers and node.

Save, read and update plain javascript objects into ArrayBuffer (And not only TypedArrays), using regular javascript object api, without serialization/deserialization, or pre-defined schema.
In other words, It's an implementation of javascript objects in user-land.

That's enables us to transfer or share objects in-memory with WebWorker without additional memory or serialization While the library is not 1.0, it is usable.

A core part of the library is an allocator, that allocates & free memory on the ArrayBuffer for us!
The allocator in use is @thi.ng/malloc, part of the amazing thi.ng/umbrella project

🐉🐉🐉 Adventurers Beware 🐉🐉🐉

Using this library, and workers in general, will not necessarily make you code faster.
First be sure where are your bottlenecks and if you don't have a better and more simple workaround.
I personally also really like what's going on around the main thread scheduling proposal and react userland scheduler that powers concurrent react

Quick example

import { createObjectBuffer, getUnderlyingArrayBuffer } from "@bnaya/objectbuffer";

const initialValue = {
  foo: { bar: new Date(), arr: [1], nesting:{ WorksTM: true } }
};
// ArrayBuffer is created under the hood
const myObject = createObjectBuffer(
  {},
  // size in bytes
  1024,
  initialValue
);

const arrayBuffer = getUnderlyingArrayBuffer(myObject);

myObject.additionalProp = "new Value";
myObject.arr.push(2);

Play with it (codesandbox)

• Sort Array on worker (comlink)
• Sort Array on worker (no comlink)
• Shared memory - SharedArrayBuffer

See also main.js for shared memory example. to run it: clone the repo, yarn install and yarn browser-playground

API reference

link

Why

Exchanging plain objects with WebWorkers is done by serializing and copying the data to the other side.
for some use-cases, it's slow and memory expensive.
ArrayBuffer can be transferred without a copy, and SharedArrayBuffer can be directly shared, but out of the box, it's hard to use ArrayBuffer as more than a TypedArray.

Why not FlatBuffers

For many cases FlatBuffers is the right tool!
FlatBuffers requires full re-serialization when changing values. inside. The api is also more different than javascript objects.

Disclaimer / Motivation

I'm working on it mostly from personal interest, and i'm not using it for any project yet.
Before putting any eggs in the basket, please go over the implementation details document

What's working

• strings
• number
• objects (with nesting and all)
• arrays
• Date
• BigInt
• Internal references (foo.bar2 = foo.bar will not create a copy, but a reference)
• Automatic reference counting, to dispose a value you need to use the disposeWrapperObject or to have WeakRef support
• Internal equality between objects (foo.bar === foo.bar will be true)
• global lock for shared memory using Atomics (I hope its really working)

Caveats & Limitations

• Need to specify size for the ArrayBuffer. When exceed that size, exception will be thrown. (Can be extended later with a utility function, but not automatically)
• Size must be multiplication of 8
• Set, Map, Object keys can be only string or numbers. no symbols or other things
• You can't save objects with circularities (But you can create them on objectbuffer)
• No prototype chain. no methods on the objects
• Adding getters, setters, will not work/break the library
• deleting/removing the current key of Map/Set while iteration will make you skip the next key #60
• unreliable_sizeOf is unreliable due to hashmap array side depends on actual keys, Also It's an expensive operation

What's not working yet, but can be

• bigint bigger than 64 bit

What's probably never going to work

• Anything that cannot go into JSON.stringify
• Symbol

Contribution / Collaboration

There's a huge place for optimizations, code hygiene, and features!
Feel free to open issues and maybe implementing missing parts.
The code is Written in TypeScript 🦾, but the semantics are more like C 🥵
Have a look on the issues and see if you find something interesting

If you came this far, you better also look at:

• GoogleChromeLabs/buffer-backed-object
• FlatBuffers