@ariestools/aries-datalake-client
REST clients for the Aries datalake control and data planes, with Node-only local development helpers kept behind the Node entry point.
Entry points
@ariestools/aries-datalake-client is the Node entry point. It includes the REST clients plus local filesystem-backed clients, credentials, defaults, and ARIES_HOME helpers.
@ariestools/aries-datalake-client/browser is the browser entry point. It exports REST clients, the composite client, their interfaces, the asynchronous token-supplier contract, and browser-safe datalake wire contracts.
Use an asynchronous token supplier that is invoked for each request. The application adapter may return a still-valid short-lived token from page memory or request a fresh wallet JWT through the XL1 browser wallet gateway:
import {
RestDatalakeClient,
RestPayloadsClient,
} from '@ariestools/aries-datalake-client/browser'
declare function requestWalletJwt(audience: string): Promise<string>
const authToken = async ({ audience }: { audience: string }): Promise<string> => {
return await requestWalletJwt(audience)
}
const control = new RestDatalakeClient({
baseUrl: 'https://control.example',
authToken,
})
const payloads = new RestPayloadsClient({
baseUrl: 'https://data.example',
datalakeId: 'dl_example',
authToken,
})
The supplier receives the exact control-plane audience or datalake id. Treat its return value as a sensitive, short-lived bearer token: cache it only in page memory when useful, and never persist it in browser storage or logs. The wallet gateway controls key access; the Aries HTTP services independently verify audience, signature, origin, scope, signer identity, and live ACL state.
Bounded exact-root archive
ExactRootArchive is available from both entries. Supply an explicit native SDK
root-identity callback, configured hash domain and lake, a trusted append-only
issuer, and all six positive finite limits: root count, request bytes, decoded
response bytes, token bytes, whole-operation milliseconds and concurrent work.
The client snapshots JSON bodies, verifies native roots, sends the native insert
request, then independently resolves every required root through anonymous
readback with credentials omitted and redirects rejected. Duplicate receipts
never replace readback. No write cache, queue, automatic retry or recovery journal
is introduced.
For the opt-in service-wallet profile, the issuer signs a JWT with the lake id as
its exact audience and payloadScopes: ['payload:append'], without browser
origin/scope or record/blob claims. The data plane must explicitly enable service
wallet JWTs and the signer must hold a live runner grant. A live public-viewer
grant independently permits readback. Tokens without payloadScopes retain
legacy service authority and do not qualify this restricted profile.
ExactRootArchiveError reports a secret-free code and writeState:
not-started, uncertain after dispatch, or acknowledged after an HTTP success.
Every public failure is a fresh error with a validated code and captured write
state; callback messages, causes and private receipts are discarded. A caller
timeout does not undo storage. Pending callbacks/effects retain the
concurrency slot until they settle, even after the caller receives cancellation.
Late response bodies are cancelled, and native stream cancellation cleanup is
awaited before that slot is released; hanging cleanup keeps capacity occupied.
Byte limits count UTF-8 JSON requests and decoded response streams, including
errors; raw network/decompression/header/chunk allocation limits belong to the
trusted host transport. The trusted issuer separately bounds its own transport,
returns no logged/persisted credentials and honors the supplied abort signal.
The identity callback preserves client/application fields and uses the owning
SDK projection for storage metadata; claimed hashes are never proof.
Multiple required datalakes
CompositePayloadsClient writes each target's eligible subset and verifies it by
reading the content back. Configure a complete S3 collection alongside a selected
Auto Drive collection using already authenticated RestPayloadsClient instances:
import { CompositePayloadsClient, type RestPayloadsClient } from '@ariestools/aries-datalake-client/browser'
import { asAnyPayload, PayloadBuilder } from '@xyo-network/sdk'
declare const autoDriveClient: RestPayloadsClient
declare const s3Client: RestPayloadsClient
const composite = new CompositePayloadsClient({
identify: async (payload) => {
const canonical = asAnyPayload(payload, true)
return {
hash: await PayloadBuilder.hash(canonical),
dataHash: await PayloadBuilder.dataHash(canonical),
}
},
targets: [
{
name: 'auto-drive',
client: autoDriveClient,
policy: {
mode: 'selected',
allowedSchemas: ['com.example.smallrecord'],
maxPayloadBytes: 4096,
revision: 'v1',
},
},
{ name: 's3', client: s3Client, policy: { mode: 'all' } },
],
})
const result = await composite.insertWithReceipts([
{ schema: 'com.example.smallrecord', value: 1 },
{ schema: 'com.example.other', value: 2 },
])
const read = await composite.getWithReceipts(result.acknowledged.map(payload => payload._hash))
All targets are required for their eligible subset. A selected policy requires
an explicit allowlist and positive byte limit; an empty list selects nothing.
disallowedSchemas takes precedence. schemaMaxPayloadBytes may tighten the
global byte limit, and isValid may supply a trusted structural validator.
Sizes count UTF-8 JSON including client metadata and excluding storage metadata.
The receiving service must enforce its policy independently; this client does not
authorize permanent uploads or implement service quotas.
insert(payloads) returns the verified acknowledgment array, including duplicates;
insertWithReceipts(payloads) also returns per-target eligibility and outcomes.
A rejected expected-eligible item or failed required target throws
CompositePayloadsWriteError. Its result retains verified partial success and
sanitized receipts, not raw provider errors. A complete target's success does not
satisfy a different target's obligation. Callers own durable retry state and
policy-revision coordination; this client does not resume a failed operation.
get(hashes) and getMany(hashes) return verified payload arrays.
getWithReceipts(hashes) adds source attribution. Reads try targets in declared
order for unresolved hashes only. Provider errors or corrupt responses throw
CompositePayloadsReadError; they are not silently converted into misses.
Reads never copy content between targets. Supply canonical XYO PayloadBuilder
hashes through identify; the composite package itself adds no XYO runtime
dependency to browser consumers.
Only flat payload bodies are accepted. Validate and flatten known hydrated
protocol envelopes before insertion. There is no global sequence, clear, delete,
or usage API and no automatic registration into an XL1 connection. The
payload-array insert/get methods are intended for a protocol adapter; keep
the SDK's exact transaction-content acknowledgment check in that adapter.
Successful read-back is readable storage, not proof of provider network archival.
Explicit public payload reads
Use RestPublicPayloadsReader when the provider has granted public viewer access
to the datalake. This is separate from the authenticated clients above; an
authentication failure never triggers an anonymous retry.
import { RestPublicPayloadsReader } from '@ariestools/aries-datalake-client/browser'
const reader = new RestPublicPayloadsReader({
baseUrl: 'https://data.example/plane/v1/datalakes/dl_public',
datalakeId: 'dl_public',
})
const controller = new AbortController()
const payload = await reader.get('configured-content-hash', {
maxResponseBytes: 65_536,
signal: controller.signal,
})
The reader exposes only get(hash, options?) and getMany(hashes, options?).
It has no token supplier, header hooks, listing or mutation methods. Requests
omit ambient credentials and reject redirects. Configuration accepts absolute
HTTP(S) URLs, including a shared-host path prefix, and rejects userinfo and
fragments. Local HTTP is permitted explicitly; address/DNS policy remains the
host's responsibility. An optional fetchImpl is a trusted transport override
that must retain those policies. Without it, fetch is resolved at request time.
maxResponseBytes is an optional positive safe integer. It counts decoded Fetch
response-stream bytes before JSON parsing, on success and error responses,
including the whole native object/array and its metadata. Omission means no size
limit. It does not bound raw compressed traffic, headers, decompression work or
connection buffers. The signal is forwarded to fetch and remains active during
body consumption; aborting does not wait indefinitely for stream cleanup.
Successful reads validate the stored-payload shape and preserve all fields.
Neither requested paths nor _hash/_dataHash metadata verify content identity;
callers that need content assurance must hash and validate the returned data.
Transport, size, cancellation, malformed-response and access failures remain
failed reads, not evidence that an advertised object never existed.
Public viewer access applies to the whole datalake, including provider-side
enumeration and usage APIs. Prefer a dedicated public viewer lake with
authenticated writers; public runner access also permits mutations. New lakes
are private until granted access, and browser access additionally requires the
provider's CORS configuration. This client does not change ACLs or retention.
The public reader requires the released @ariestools/sdk 8.2.0 or a compatible
8.2 patch, plus its required @opentelemetry/api peer. It does not require zod
for these reads; other SDK features and datalake peers can have separate needs.