Huge News!Announcing our $40M Series B led by Abstract Ventures.Learn More →
Socket
Sign inDemoInstall
Socket
Sign inDemoInstall

@agoric/marshal

Package Overview
Dependencies
Maintainers
5
Versions
156
Alerts
File Explorer

Advanced tools

License
Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

@agoric/marshal - npm Package Compare versions

Comparing version 0.3.2 to 0.4.0

src/extra-types.d.ts
src/ibidTables.js
src/marshal-justin.js

23

CHANGELOG.md

@@ -6,2 +6,25 @@ # Change Log

# [0.4.0](https://github.com/Agoric/agoric-sdk/compare/@agoric/marshal@0.3.2...@agoric/marshal@0.4.0) (2021-03-16)
### Bug Fixes
* fix ibids. test ibids and slots ([#2625](https://github.com/Agoric/agoric-sdk/issues/2625)) ([891d9fd](https://github.com/Agoric/agoric-sdk/commit/891d9fd236ca86b63947384064b675c52e960abd))
* make separate 'test:xs' target, remove XS from 'test' target ([b9c1a69](https://github.com/Agoric/agoric-sdk/commit/b9c1a6987093fc8e09e8aba7acd2a1618413bac8)), closes [#2647](https://github.com/Agoric/agoric-sdk/issues/2647)
* **marshal:** add Data marker, tolerate its presence ([d7b190f](https://github.com/Agoric/agoric-sdk/commit/d7b190f340ba336bd0d76a2ca8ed4829f227be61))
* **marshal:** add placeholder warnings ([8499b8e](https://github.com/Agoric/agoric-sdk/commit/8499b8e4584f3ae155913f95614980a483c487e2))
* **marshal:** serialize empty objects as data, not pass-by-reference ([aeee1ad](https://github.com/Agoric/agoric-sdk/commit/aeee1adf561d44ed3bc738989be605b683b3b656)), closes [#2018](https://github.com/Agoric/agoric-sdk/issues/2018)
* separate ibid tables ([#2596](https://github.com/Agoric/agoric-sdk/issues/2596)) ([e0704eb](https://github.com/Agoric/agoric-sdk/commit/e0704eb640a54ceec11b39fc924488108cb10cee))
### Features
* **marshal:** add Data() to all unserialized empty records ([946fd6f](https://github.com/Agoric/agoric-sdk/commit/946fd6f1b811c55ee39668100755db24f1b52329))
* **marshal:** allow marshalSaveError function to be specified ([c93bb04](https://github.com/Agoric/agoric-sdk/commit/c93bb046aecf476dc9ccc537671a14f446b89ed4))
* **marshal:** Data({}) is pass-by-copy ([03d7b5e](https://github.com/Agoric/agoric-sdk/commit/03d7b5eed8ecd3f24725d6ea63919f4398d8a2f8))
## [0.3.2](https://github.com/Agoric/agoric-sdk/compare/@agoric/marshal@0.3.1...@agoric/marshal@0.3.2) (2021-02-22)

@@ -8,0 +31,0 @@

1

index.js

@@ -12,4 +12,5 @@ export {

Far,
Data,
} from './src/marshal';
export { stringify, parse } from './src/marshal-stringify';

15

package.json
{
"name": "@agoric/marshal",
"version": "0.3.2",
"version": "0.4.0",
"description": "marshal",

@@ -16,2 +16,3 @@ "parsers": {

"test:nyc": "nyc ava",
"test:xs": "exit 0",
"pretty-fix": "prettier --write '**/*.js'",

@@ -39,9 +40,9 @@ "pretty-check": "prettier --check '**/*.js'",

"dependencies": {
"@agoric/assert": "^0.2.2",
"@agoric/eventual-send": "^0.13.2",
"@agoric/nat": "^2.0.1",
"@agoric/promise-kit": "^0.2.2"
"@agoric/assert": "^0.2.3",
"@agoric/eventual-send": "^0.13.3",
"@agoric/nat": "^4.0.0",
"@agoric/promise-kit": "^0.2.3"
},
"devDependencies": {
"@agoric/install-ses": "^0.5.2",
"@agoric/install-ses": "^0.5.3",
"ava": "^3.12.1",

@@ -77,3 +78,3 @@ "esm": "^3.2.25",

},
"gitHead": "d9b77a4d36fe99c963e86733b90d878231201422"
"gitHead": "5ad5f483f05ba20d903b4423bfb4fa0d2ce75fd7"
}

370

src/marshal.js

@@ -6,5 +6,6 @@ // @ts-check

import Nat from '@agoric/nat';
import { Nat } from '@agoric/nat';
import { assert, details as X, q } from '@agoric/assert';
import { isPromise } from '@agoric/promise-kit';
import { makeReplacerIbidTable, makeReviverIbidTable } from './ibidTables';

@@ -16,2 +17,3 @@ import './types';

setPrototypeOf,
create,
getOwnPropertyDescriptors,

@@ -252,2 +254,31 @@ defineProperties,

/**
* Everything having to do with a `dataProto` is a temporary kludge until
* we're on the other side of #2018. TODO remove.
*/
const dataProto = harden(
create(objectPrototype, {
[PASS_STYLE]: { value: 'copyRecord' },
}),
);
const isDataProto = proto => {
if (!isFrozen(proto)) {
return false;
}
if (getPrototypeOf(proto) !== objectPrototype) {
return false;
}
const {
// @ts-ignore
[PASS_STYLE]: passStyleDesc,
...rest
} = getOwnPropertyDescriptors(proto);
return (
passStyleDesc &&
passStyleDesc.value === 'copyRecord' &&
ownKeys(rest).length === 0
);
};
/**
* @param {Passable} val

@@ -257,3 +288,4 @@ * @returns {boolean}

function isPassByCopyRecord(val) {
if (getPrototypeOf(val) !== objectPrototype) {
const proto = getPrototypeOf(val);
if (proto !== objectPrototype && !isDataProto(proto)) {
return false;

@@ -263,8 +295,3 @@ }

const descKeys = ownKeys(descs);
if (descKeys.length === 0) {
// empty non-array objects are pass-by-remote, not pass-by-copy
// TODO Beware: Unmarked empty records will become pass-by-copy
// See https://github.com/Agoric/agoric-sdk/issues/2018
return false;
}
for (const descKey of descKeys) {

@@ -307,10 +334,20 @@ if (typeof descKey === 'symbol') {

const toString = () => `[${allegedName}]`;
return harden({
__proto__: oldProto,
[PASS_STYLE]: REMOTE_STYLE,
toString,
[Symbol.toStringTag]: allegedName,
});
return harden(
create(oldProto, {
[PASS_STYLE]: { value: REMOTE_STYLE },
toString: { value: toString },
[Symbol.toStringTag]: { value: allegedName },
}),
);
};
/**
* Throw if val is not the correct shape for the prototype of a Remotable.
*
* TODO: It would be nice to typedef this shape and then declare that this
* function asserts it, but we can't declare a type with PASS_STYLE from JSDoc.
*
* @param {{ [PASS_STYLE]: string, [Symbol.toStringTag]: string, toString: () =>
* void}} val the value to verify
*/
const assertRemotableProto = val => {

@@ -328,7 +365,5 @@ assert.typeof(val, 'object', X`cannot serialize non-objects like ${val}`);

const {
// @ts-ignore
[PASS_STYLE]: { value: passStyleValue },
// @ts-ignore
toString: { value: toStringValue },
// @ts-ignore
// @ts-ignore https://github.com/microsoft/TypeScript/issues/1863
[Symbol.toStringTag]: { value: toStringTagValue },

@@ -366,7 +401,8 @@ ...rest

assert(
// @ts-ignore
!('get' in descs[key]),
// Typecast needed due to https://github.com/microsoft/TypeScript/issues/1863
!('get' in descs[/** @type {string} */ (key)]),
X`cannot serialize objects with getters like ${q(String(key))} in ${val}`,
);
assert.typeof(
// @ts-ignore https://github.com/microsoft/TypeScript/issues/1863
val[key],

@@ -378,2 +414,6 @@ 'function',

);
assert(
key !== PASS_STYLE,
X`A pass-by-remote cannot shadow ${q(PASS_STYLE)}`,
);
});

@@ -477,2 +517,4 @@ }

assertRemotable(val);
// console.log(`--- @@marshal: pass-by-ref object without Far/Remotable`);
// assert.fail(X`pass-by-ref object without Far/Remotable`);
return REMOTE_STYLE;

@@ -498,75 +540,2 @@ }

/**
* The ibid logic relies on
* * JSON.stringify on an array visiting array indexes from 0 to
* arr.length -1 in order, and not visiting anything else.
* * JSON.parse of a record (a plain object) creating an object on
* which a getOwnPropertyNames will enumerate properties in the
* same order in which they appeared in the parsed JSON string.
*/
function makeReplacerIbidTable() {
const ibidMap = new Map();
let ibidCount = 0;
return harden({
has(obj) {
return ibidMap.has(obj);
},
get(obj) {
return ibidMap.get(obj);
},
add(obj) {
ibidMap.set(obj, ibidCount);
ibidCount += 1;
},
});
}
function makeReviverIbidTable(cyclePolicy) {
const ibids = [];
const unfinishedIbids = new WeakSet();
return harden({
get(allegedIndex) {
const index = Nat(allegedIndex);
assert(index < ibids.length, X`ibid out of range: ${index}`, RangeError);
const result = ibids[index];
if (unfinishedIbids.has(result)) {
switch (cyclePolicy) {
case 'allowCycles': {
break;
}
case 'warnOfCycles': {
console.log(`Warning: ibid cycle at ${index}`);
break;
}
case 'forbidCycles': {
assert.fail(X`Ibid cycle at ${q(index)}`, TypeError);
}
default: {
assert.fail(
X`Unrecognized cycle policy: ${q(cyclePolicy)}`,
TypeError,
);
}
}
}
return result;
},
register(obj) {
ibids.push(obj);
return obj;
},
start(obj) {
ibids.push(obj);
unfinishedIbids.add(obj);
return obj;
},
finish(obj) {
unfinishedIbids.delete(obj);
return obj;
},
});
}
/**
* Special property name that indicates an encoding that needs special

@@ -596,3 +565,10 @@ * decoding.

convertSlotToVal = defaultSlotToValFn,
{ marshalName = 'anon-marshal', errorTagging = 'on' } = {},
{
marshalName = 'anon-marshal',
errorTagging = 'on',
// We prefer that the caller instead log to somewhere hidden
// to be revealed when correlating with the received error.
marshalSaveError = err =>
console.log('Temporary logging of sent error', err),
} = {},
) {

@@ -624,2 +600,3 @@ assert.typeof(marshalName, 'string');

slotIndex = slotMap.get(val);
assert.typeof(slotIndex, 'number');
} else {

@@ -675,8 +652,2 @@ const slot = convertValToSlot(val);

/**
* Just consists of data that rounds trips to plain data.
*
* @typedef {any} PlainJSONData
*/
/**
* Must encode `val` into plain JSON data *canonically*, such that

@@ -692,3 +663,3 @@ * `sameStructure(v1, v2)` implies

* @param {Passable} val
* @returns {PlainJSONData}
* @returns {Encoding}
*/

@@ -747,5 +718,7 @@ const encode = val => {

// Backreference to prior occurrence
const index = ibidTable.get(val);
assert.typeof(index, 'number');
return harden({
[QCLASS]: 'ibid',
index: ibidTable.get(val),
index,
});

@@ -796,8 +769,3 @@ }

assert.note(val, X`Sent as ${errorId}`);
// TODO we need to instead log to somewhere hidden
// to be revealed when correlating with the received error.
// By sending this to `console.log`, under swingset this is
// enabled by `agoric start --reset -v` and not enabled without
// the `-v` flag.
console.log('Temporary logging of sent error', val);
marshalSaveError(val);
return harden({

@@ -846,36 +814,40 @@ [QCLASS]: 'error',

// We stay close to the algorithm at
// https://tc39.github.io/ecma262/#sec-json.parse , where
// fullRevive(JSON.parse(str)) is like JSON.parse(str, revive))
// for a similar reviver. But with the following differences:
//
// Rather than pass a reviver to JSON.parse, we first call a plain
// (one argument) JSON.parse to get rawTree, and then post-process
// the rawTree with fullRevive. The kind of revive function
// handled by JSON.parse only does one step in post-order, with
// JSON.parse doing the recursion. By contrast, fullParse does its
// own recursion, enabling it to interpret ibids in the same
// pre-order in which the replacer visited them, and enabling it
// to break cycles.
//
// In order to break cycles, the potentially cyclic objects are
// not frozen during the recursion. Rather, the whole graph is
// hardened before being returned. Error objects are not
// potentially recursive, and so may be harmlessly hardened when
// they are produced.
//
// fullRevive can produce properties whose value is undefined,
// which a JSON.parse on a reviver cannot do. If a reviver returns
// undefined to JSON.parse, JSON.parse will delete the property
// instead.
//
// fullRevive creates and returns a new graph, rather than
// modifying the original tree in place.
//
// fullRevive may rely on rawTree being the result of a plain call
// to JSON.parse. However, it *cannot* rely on it having been
// produced by JSON.stringify on the replacer above, i.e., it
// cannot rely on it being a valid marshalled
// representation. Rather, fullRevive must validate that.
return function fullRevive(rawTree) {
/**
* We stay close to the algorithm at
* https://tc39.github.io/ecma262/#sec-json.parse , where
* fullRevive(harden(JSON.parse(str))) is like JSON.parse(str, revive))
* for a similar reviver. But with the following differences:
*
* Rather than pass a reviver to JSON.parse, we first call a plain
* (one argument) JSON.parse to get rawTree, and then post-process
* the rawTree with fullRevive. The kind of revive function
* handled by JSON.parse only does one step in post-order, with
* JSON.parse doing the recursion. By contrast, fullParse does its
* own recursion, enabling it to interpret ibids in the same
* pre-order in which the replacer visited them, and enabling it
* to break cycles.
*
* In order to break cycles, the potentially cyclic objects are
* not frozen during the recursion. Rather, the whole graph is
* hardened before being returned. Error objects are not
* potentially recursive, and so may be harmlessly hardened when
* they are produced.
*
* fullRevive can produce properties whose value is undefined,
* which a JSON.parse on a reviver cannot do. If a reviver returns
* undefined to JSON.parse, JSON.parse will delete the property
* instead.
*
* fullRevive creates and returns a new graph, rather than
* modifying the original tree in place.
*
* fullRevive may rely on rawTree being the result of a plain call
* to JSON.parse. However, it *cannot* rely on it having been
* produced by JSON.stringify on the replacer above, i.e., it
* cannot rely on it being a valid marshalled
* representation. Rather, fullRevive must validate that.
*
* @param {Encoding} rawTree must be hardened
*/
function fullRevive(rawTree) {
if (Object(rawTree) !== rawTree) {

@@ -885,2 +857,5 @@ // primitives pass through

}
// Assertions of the above to narrow the type.
assert.typeof(rawTree, 'object');
assert(rawTree !== null);
if (QCLASS in rawTree) {

@@ -893,3 +868,7 @@ const qclass = rawTree[QCLASS];

);
switch (qclass) {
assert(!Array.isArray(rawTree));
// Switching on `encoded[QCLASS]` (or anything less direct, like
// `qclass`) does not discriminate rawTree in typescript@4.2.3 and
// earlier.
switch (rawTree['@qclass']) {
// Encoding of primitives not handled by JSON

@@ -909,9 +888,9 @@ case 'undefined': {

case 'bigint': {
const { digits } = rawTree;
assert.typeof(
rawTree.digits,
digits,
'string',
X`invalid digits typeof ${q(typeof rawTree.digits)}`,
X`invalid digits typeof ${q(typeof digits)}`,
);
/* eslint-disable-next-line no-undef */
return BigInt(rawTree.digits);
return BigInt(digits);
}

@@ -923,22 +902,24 @@ case '@@asyncIterator': {

case 'ibid': {
return ibidTable.get(rawTree.index);
const { index } = rawTree;
return ibidTable.get(index);
}
case 'error': {
const { name, message, errorId } = rawTree;
assert.typeof(
rawTree.name,
name,
'string',
X`invalid error name typeof ${q(typeof rawTree.name)}`,
X`invalid error name typeof ${q(typeof name)}`,
);
assert.typeof(
rawTree.message,
message,
'string',
X`invalid error message typeof ${q(typeof rawTree.message)}`,
X`invalid error message typeof ${q(typeof message)}`,
);
const EC = getErrorConstructor(`${rawTree.name}`) || Error;
const error = harden(new EC(`${rawTree.message}`));
const EC = getErrorConstructor(`${name}`) || Error;
const error = harden(new EC(`${message}`));
ibidTable.register(error);
if (typeof rawTree.errorId === 'string') {
if (typeof errorId === 'string') {
// errorId is a late addition so be tolerant of its absence.
assert.note(error, X`Received as ${rawTree.errorId}`);
assert.note(error, X`Received as ${errorId}`);
}

@@ -949,7 +930,9 @@ return error;

case 'slot': {
const slot = slots[Nat(rawTree.index)];
return ibidTable.register(convertSlotToVal(slot, rawTree.iface));
const { index, iface } = rawTree;
const slot = slots[Number(Nat(index))];
return ibidTable.register(convertSlotToVal(slot, iface));
}
case 'hilbert': {
const { original, rest } = rawTree;
assert(

@@ -960,5 +943,9 @@ 'original' in rawTree,

const result = ibidTable.start({});
result[QCLASS] = fullRevive(rawTree.original);
result[QCLASS] = fullRevive(original);
if ('rest' in rawTree) {
const rest = fullRevive(rawTree.rest);
assert(
rest !== undefined,
X`Rest encoding must not be undefined`,
);
const restObj = fullRevive(rest);
// TODO really should assert that `passStyleOf(rest)` is

@@ -968,6 +955,6 @@ // `'copyRecord'` but we'd have to harden it and it is too

assert(
!(QCLASS in rest),
!(QCLASS in restObj),
X`Rest must not contain its own definition of ${q(QCLASS)}`,
);
defineProperties(result, getOwnPropertyDescriptors(rest));
defineProperties(result, getOwnPropertyDescriptors(restObj));
}

@@ -982,5 +969,5 @@ return ibidTable.finish(result);

} else if (Array.isArray(rawTree)) {
const { length } = rawTree;
const result = ibidTable.start([]);
const len = rawTree.length;
for (let i = 0; i < len; i += 1) {
for (let i = 0; i < length; i += 1) {
result[i] = fullRevive(rawTree[i]);

@@ -990,11 +977,21 @@ }

} else {
const result = ibidTable.start({});
let result = ibidTable.start({});
const names = ownKeys(rawTree);
for (const name of names) {
assert.typeof(name, 'string');
result[name] = fullRevive(rawTree[name]);
if (names.length === 0) {
// eslint-disable-next-line no-use-before-define
result = Data(result);
} else {
for (const name of names) {
assert.typeof(
name,
'string',
X`Property ${name} of ${rawTree} must be a string`,
);
result[name] = fullRevive(rawTree[name]);
}
}
return ibidTable.finish(result);
}
};
}
return fullRevive;
}

@@ -1047,3 +1044,3 @@

*/
function Remotable(iface = 'Remotable', props = undefined, remotable = {}) {
const Remotable = (iface = 'Remotable', props = undefined, remotable = {}) => {
// TODO unimplemented

@@ -1078,2 +1075,4 @@ assert.typeof(

);
// Ensure that the remotable isn't already frozen.
assert(!isFrozen(remotable), X`Remotable ${remotable} is already frozen`);
const remotableProto = makeRemotableProto(

@@ -1103,3 +1102,3 @@ getPrototypeOf(remotable),

return remotable;
}
};

@@ -1122,1 +1121,32 @@ harden(Remotable);

export { Far };
/**
* Everything having to do with `Data` is a temporary kludge until
* we're on the other side of #2018. TODO remove.
*
* @param {Object} record
*/
const Data = record => {
// Ensure that the record isn't already marked.
assert(
!(PASS_STYLE in record),
X`Record ${record} is already marked as a ${q(record[PASS_STYLE])}`,
);
// Ensure that the record isn't already frozen.
assert(!isFrozen(record), X`Record ${record} is already frozen`);
assert(
getPrototypeOf(record) === objectPrototype,
X`A record ${record} must initially inherit from Object.prototype`,
);
setPrototypeOf(record, dataProto);
harden(record);
assert(
isPassByCopyRecord(record),
X`Data() can only be applied to otherwise pass-by-copy records`,
);
return record;
};
harden(Data);
export { Data };

50

src/types.js

@@ -0,1 +1,4 @@

// eslint-disable-next-line spaced-comment
/// <reference path="extra-types.d.ts" />
/**

@@ -58,3 +61,3 @@ * @typedef { "bigint" | "boolean" | "null" | "number" | "string" | "symbol" | "undefined" | "copyArray" | "copyRecord" | "copyError" | "promise" | "presence" } PassStyle

/**
* @typedef {SOMETHING} Remotable
* @typedef {*} Remotable
* Might be an object explicitly deemed to be `Remotable`, an object inferred

@@ -87,24 +90,26 @@ * to be Remotable, or a remote presence of a Remotable.

/**
* @typedef Encoding
* @template T
* @typedef {{ '@qclass': T }} EncodingClass
*/
/**
* @typedef {EncodingClass<'NaN'> |
* EncodingClass<'undefined'> |
* EncodingClass<'Infinity'> |
* EncodingClass<'-Infinity'> |
* EncodingClass<'bigint'> & { digits: string } |
* EncodingClass<'@@asyncIterator'> |
* EncodingClass<'ibid'> & { index: number } |
* EncodingClass<'error'> & { name: string, message: string, errorId?: string } |
* EncodingClass<'slot'> & { index: number, iface?: InterfaceSpec } |
* EncodingClass<'hilbert'> & { original: Encoding, rest?: Encoding }} EncodingUnion
* @typedef {{ [index: string]: Encoding, '@qclass'?: undefined }} EncodingRecord
* We exclude '@qclass' as a property in encoding records.
* @typedef {EncodingUnion | null | string | boolean | number | EncodingRecord} EncodingElement
*/
/**
* @typedef {EncodingElement | NestedArray<EncodingElement>} Encoding
* The JSON structure that the data portion of a Passable serializes to.
*
* TODO turn into a discriminated union type
* { [QCLASS]: 'undefined' }
* | { [QCLASS]: 'NaN' }
* | { [QCLASS]: 'Infinity' }
* | { [QCLASS]: '-Infinity' }
* | { [QCLASS]: 'bigint', digits: string }
* // Likely to generalize to more symbols
* | { [QCLASS]: '@@asyncIterator' }
* // Should be path rather than index
* | { [QCLASS]: 'ibid', index: number }
* | { [QCLASS]: 'error', name: string, message: string, errorId? string }
* | { [QCLASS]: 'slot', index: number, iface? InterfaceSpec }
* | { [QCLASS]: 'hilbert', original: Encoding, rest? Record<string, Encoding> }
* // Primitive values directly encodable in JSON
* | null | string | boolean | number
* | Encoding[]
* // excluding QCLASS as a property name
* | Record<string, Encoding>
*
* The QCLASS 'hilbert' is a reference to the Hilbert Hotel

@@ -162,3 +167,4 @@ * of https://www.ias.edu/ideas/2016/pires-hilbert-hotel

* @property {string=} marshalName
* @property {('on'|'off')=} errorTagging
* @property {'on'|'off'=} errorTagging
* @property {(err: Error) => void=} marshalSaveError
*/

@@ -165,0 +171,0 @@

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

About

Packages

npm

Go

Maven

PyPI

Rubygems

Stay in touch

Get open source security insights delivered straight into your inbox.

  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc