@pdpp/connector-protocol

@pdpp/connector-protocol

The connector authoring contract for PDPP (the Personal Data Portability Protocol): the JSONL wire-protocol message types, the LocalCollectorDefinition type contract, the bootstrap/emit/scope-filter primitives connector authors import directly, and small text/auth/retry utilities connectors depend on.

Bottom of the dependency graph — this package depends on nothing in @pdpp/collector-runtime or @pdpp/polyfill-connectors, so connector authors never pull in the runtime package or its release cadence just to get the files they author against.

Entry points

• . — the top-level connector-authoring types and primitives
• ./auth — authentication helpers connectors use against upstream providers
• ./collector-definition — the LocalCollectorDefinition contract
• ./connector-runtime-protocol — the JSONL wire-protocol message types shared with @pdpp/collector-runtime
• ./http-retry — retry-after/backoff helpers for HTTP-based connectors
• ./pdpp-safe-text — text-safety helpers for connector-authored content
• ./safe-text-preview — bounded text previews for connector-authored content

Status

Published from PDP-Connect/data-connect, under packages/connector-protocol.

Protocol rollout

Protocol version 0.0.3 adds the BLOB event and the BLOB protocol capability. A connector declaring that capability writes and closes a regular JSON file in the host-provided spool, then emits a BLOB descriptor immediately before the matching RECORD. The host validates the descriptor, bytes, and record binding before accepting that record. validateHostBlobMessage checks the wire descriptor at the untyped boundary. recordKeysEqual defines binding: scalar and compound keys are distinct, and compound member order matters. A runtime must advertise BLOB only after it can durably ingest the bytes. The signed profile's host_blob_write declaration separately limits streams, spool size, and host permission; protocol capability negotiation does not grant it. The local collector currently does not advertise it and rejects any BLOB it sees. Merging this PR would trigger a semantic-release minor publication; the consumer runtime must be ready before merge.

Protocol version 0.0.2 (the CONNECTOR_PROTOCOL_VERSION constant exported from connector-runtime-protocol.ts, tracked independently of this package's own release version) adds the STREAM_EVIDENCE wire message. It is a breaking wire addition for fail-closed runtimes that do not recognize that message, so the runtime must be upgraded before a connector that emits it is distributed.

STREAM_EVIDENCE carries considered (the full count of keys a state_stream-declared stream's own hydration lane considered this run) and outcomes: { emitted, unchanged, gapped, unaccounted } — a disjoint partition of those considered keys, not a scalar covered count. There is no covered field on the wire: a single count can never distinguish "emitted", "unchanged", "gapped", and "unaccounted" keys, so a reader that wants a covered projection derives it as outcomes.emitted + outcomes.unchanged, deliberately excluding gapped and unaccounted. outcomes.emitted + outcomes.unchanged + outcomes.gapped + outcomes.unaccounted MUST equal considered exactly; every count field is capped at Number.MAX_SAFE_INTEGER. Call validateStreamEvidenceCounts at the untyped wire boundary (immediately after JSON.parse) to enforce the bounds and the sum check — the type system alone cannot, since the value arrives as unknown JSON.

Connector	Runtime	Result
old connector	new runtime	compatible; no new message is emitted
new connector emitting STREAM_EVIDENCE	old fail-closed runtime	incompatible; pre-spawn gate rejects it
new connector	new runtime advertising STREAM_EVIDENCE	compatible

Connectors declare required protocol capabilities with protocol_capabilities. The runtime advertises protocolVersion and protocolCapabilities, and rejects missing capabilities before spawning a connector. SKIP_RESULT.boundary_claim is optional, does not require a protocol capability, and is disclosure-only: it never affects, reduces, or excludes anything from the servable denominator, coverage calculation, or connection/system health rollup, even when an independently recorded coverage horizon agrees with it.

The committed artifact.json binds the package version and source-input digest to a reproducible npm tarball digest. Run npm run artifact:verify after any protocol or package build change. Run npm run artifact:generate when the expected artifact metadata must be regenerated.