@sitedex/web-bot-auth-lint

@sitedex/web-bot-auth-lint

Grade a published Web Bot Auth key directory for the mistakes that quietly break verification: an expired key, the wrong curve, a signature a verifier will reject, or a private signing key left in a file the whole internet can read. Point it at a /.well-known/http-message-signatures-directory and get back a graded report of what a verifier will reject.

For anyone who publishes a directory so crawlers can be recognized, or who audits someone else's.

Zero dependencies. Node 20+, or any runtime with Web Crypto (browsers, Cloudflare Workers, Deno). Apache-2.0.

Linting vs. signing/verifying

cloudflare/web-bot-auth signs and verifies Web Bot Auth requests. This package lints the directory you publish. It reads the JWK Set at your well-known path, checks the keys and the signature on the response, and reports what would stop a verifier from accepting it.

Standards lineage

The rules come from four places. The linter keeps them straight and tags each finding with its source (see the output model).

• RFC 9421 — HTTP Message Signatures. The general IETF standard for signing an HTTP message. Not bot-specific; it's the mechanism underneath everything else.
• The directory draft (draft-meunier-http-message-signatures-directory). Its own IETF draft. Defines the /.well-known/http-message-signatures-directory endpoint: you publish public keys as a JWK Set and sign the directory response.
• Web Bot Auth (draft-meunier-web-bot-auth-architecture). The application. An agent proves who it is by signing its requests with a key from that directory.
• The JWK family: RFC 7517 (JWK), 7518 (JWA), 7638 (JWK Thumbprint), 8037 (Ed25519 in JOSE).

Ed25519-only is not in the spec. The architecture draft ships RSA-PSS test vectors and only forbids a shared HMAC secret. Ed25519-only and HTTPS-only are constraints of the deployed verifier, chiefly Cloudflare, which accepts any valid Ed25519 key in your directory. Cloudflare and Akamai both implement the drafts; nobody owns them. So the linter attributes those two rules to the verifier, not the spec, and says so in every finding.

Install

npm i @sitedex/web-bot-auth-lint

Quickstart

The library does not fetch. You fetch the directory and its body; validateDirectory grades what you pass it. That keeps it dependency-free and lets you validate a captured directory offline.

The easy path: directoryInputFromResponse reads the response headers for you, including the signature semantics (see below).

import { validateDirectory, directoryInputFromResponse } from '@sitedex/web-bot-auth-lint';

const dirUrl = 'https://example.com/.well-known/http-message-signatures-directory';
const res = await fetch(dirUrl);

const report = await validateDirectory(directoryInputFromResponse(res, await res.text()));

console.log(report.grade); // 'pass' | 'fail'
for (const f of report.findings) {
  console.log(`[${f.grade}] ${f.check}: ${f.detail}`);
}

Fail CI on a broken directory:

if (report.grade === 'fail') process.exit(1);

validateDirectory never throws on directory content. Malformed JSON, a missing keys array, or garbage input become a fail finding (e.g. json_parse), not an exception, so a broken directory is still a graded report. You handle fetch rejections yourself, since the library doesn't fetch.

Building the input yourself

Have the pieces already (a proxied fetch, a stored response, a fixture)? Build the DirectoryInput directly:

import { validateDirectory, type DirectoryInput } from '@sitedex/web-bot-auth-lint';

const input: DirectoryInput = {
  rawBody: body,
  status: 200,
  contentType: 'application/http-message-signatures-directory+json',
  cacheControl: 'max-age=86400',
  finalUrl: dirUrl,
  redirected: false,

  // Signature fields are optional, and the undefined-vs-null distinction is
  // load-bearing:
  //   • omit them entirely (undefined) → the linter runs NO signature checks
  //     (you're grading only the JWKS document).
  //   • pass null (you looked for the header and it was absent) → the linter
  //     warns that the response is unsigned.
  signature: null,
  signatureInput: null,
  contentDigest: null,
  requestMethod: 'GET',
};

const report = await validateDirectory(input);

directoryInputFromResponse gets this right for you: an absent Signature header comes back as null (you looked, it's unsigned), so you get the unsigned-directory warning automatically.

CLI

For a 30-second check without writing code, run it with npx. The CLI is the one piece that fetches (the library never does), so this is the whole thing:

npx @sitedex/web-bot-auth-lint example.com

Pass a bare domain and it expands to the well-known directory URL. Pass a full URL and it fetches that as-is. It prints a report grouped by authority, colorized by grade.

--json prints the raw DirectoryReport for scripting or piping into jq:

npx @sitedex/web-bot-auth-lint example.com --json

The exit code is the CI contract: 0 when the directory passes, 1 when it fails, 2 on a usage or fetch error. Color follows your terminal, and turns off when the output is piped or when NO_COLOR is set. The fetch is a well-behaved one: a 10-second timeout, a capped body read, and a descriptive User-Agent.

The output model

Every finding has two independent axes:

• grade (pass | warn | fail): how bad it is.
• authority: who says so. Lets you separate a real spec violation from Cloudflare's house rules from our own advice.

A fail under spec-must is a genuine standards violation. The same fail under linter-opinion is just advice.

interface DirectoryReport {
  grade: 'pass' | 'fail';   // fail if ANY finding failed; warns don't demote it
  findings: Finding[];
  keys: KeySummary[];
  fetchedAt: string;        // ISO timestamp
}

interface Finding {
  check: string;            // stable id, e.g. 'private_key_material'
  grade: 'pass' | 'warn' | 'fail';
  detail: string;           // human-readable; safe to render as a text node
  authority: Authority;
  specRef?: string;         // citation token, e.g. 'RFC 8037 §2'
  hero?: boolean;           // true only on the leaked-private-key finding
}

interface KeySummary {
  kid: string | null;
  crv: string | null;
  kty: string | null;
  daysToExpiry: number | null;  // floored; null when the key has no `exp`
  thumbprintMatches: boolean;   // does kid equal the key's RFC 7638 thumbprint
}

Only a fail finding demotes the report; a warn-only directory still passes. specRef resolves to a datatracker URL via the exported specRefUrl().

Example report (pass with warnings)

A valid Ed25519 key that expires soon, served without a Cache-Control header. The directory still verifies, so grade is 'pass', with two warn findings (passing findings trimmed):

{
  "grade": "pass",
  "fetchedAt": "2026-07-02T14:07:00.000Z",
  "keys": [
    {
      "kid": "NzbLsXh8uDCcd-6MNwXF4W_7noWXFZAfHkxZsRGC9Xs",
      "crv": "Ed25519",
      "kty": "OKP",
      "daysToExpiry": 9,
      "thumbprintMatches": true
    }
  ],
  "findings": [
    // ...passing checks (https, reachable, x_valid, private_key_material, ...) omitted
    {
      "check": "exp_hygiene",
      "grade": "warn",
      "detail": "Key expires soon: NzbLsXh8uDCcd-6MNwXF4W_7noWXFZAfHkxZsRGC9Xs (9 day(s)).",
      "authority": "linter-opinion",
      "specRef": "draft-meunier-http-message-signatures-directory-05 §3"
    },
    {
      "check": "cache_control",
      "grade": "warn",
      "detail": "No Cache-Control header. Add one so verifiers can cache the directory instead of refetching every time.",
      "authority": "linter-opinion"
    }
  ]
}

The authority axis

authority	Meaning
spec-must	A MUST in RFC 9421, the JWK RFCs, or the directory draft. Breaking it is a genuine violation.
spec-should	A SHOULD in those same standards. Worth following, not strictly required.
verifier-requirement	Not in any spec. Enforced by the verifier that reads your directory, chiefly Cloudflare (Ed25519-only, HTTPS-only).
linter-opinion	Our own hardening advice. A suggestion.

Presentational helpers

These ship with the package so a web UI, a CLI, and an OG-card renderer render the same words:

• CHECK_LABELS: Record<string, string> — each check id to a short label (private_key_material → "Private key material").
• AUTHORITY_LABELS: Record<Authority, { label: string; blurb: string }> — each authority to a heading and one line of context. Read AUTHORITY_LABELS[f.authority].label, not the object itself.
• AUTHORITY_ORDER: Authority[] — the four authorities, most-authoritative first, for rendering grouped sections in a stable order.
• DIRECTORY_SIGNATURE_TAG — the tag string a directory-response signature must carry (what signature_tag checks for).

The checks

Every check id, grouped. ✱ marks a sibling-aware check: Cloudflare uses any valid Ed25519 key it finds, so one bad key beside a usable one is a warn. It fails only when no usable key is left.

Transport & document

Check id	What it checks	Worst grade	Authority
https	Served over HTTPS	fail	verifier-requirement
reachable	Returns HTTP 200	fail	verifier-requirement
not_redirected	Served directly, no redirect	warn	linter-opinion
well_known_path	Lives at the well-known path	warn	verifier-requirement
content_type	Correct media type (…-directory+json)	warn	spec-must
json_parse	Body is valid JSON	fail	spec-must
keys_present	Has a usable keys array	fail	spec-must
malformed_entries	Every keys entry is a JWK object	warn	spec-should
keys_capped	Key count within the linter's evaluation cap	warn	linter-opinion

Key hygiene

Check id	What it checks	Worst grade	Authority
kty_okp	Key type is OKP ✱	fail	verifier-requirement
crv_ed25519	Curve is Ed25519 ✱	fail	verifier-requirement
x_valid	Public key is a 32-byte Ed25519 value ✱	fail	spec-must
thumbprint_kid	kid equals the key's RFC 7638 thumbprint (its fingerprint)	warn	linter-opinion
unique_kids	Key IDs are distinct within the set	warn	spec-should
cache_control	Cache-Control lets a verifier cache the directory	warn	linter-opinion
nbf_hygiene	No usable key is stuck before its not-before time ✱	fail	spec-should
exp_hygiene	No usable key is expired, and none expires within 14 days ✱	fail	spec-should / linter-opinion
private_key_material	No private signing key is published (the headline)	fail	spec-must

private_key_material is why this tool exists. If any private JWK member (d, p, q, dp, dq, qi, oth, k) appears in a published key, anyone who reads the directory can impersonate the bot. Always a hard fail, and it carries hero: true for prominent rendering.

Response signature (structural)

The shape of the signature the origin puts on the directory response. Pure, no crypto, always safe to run.

Check id	What it checks	Worst grade	Authority
response_signature	The response is signed at all	warn	verifier-requirement
signature_tag	The signature's tag is the directory tag	fail	spec-must
signature_keyid_thumbprint	The signature's key ID is the thumbprint of an advertised key	fail	spec-must

Response signature (cryptographic)

Check id	What it checks	Worst grade	Authority
signature_verified	The Ed25519 signature actually verifies	fail	verifier-requirement
signature_freshness	A valid signature hasn't expired or isn't post-dated	warn	verifier-requirement

signature_verified only emits fail on a provably bad signature: the header parsed, every covered component was rebuilt byte-for-byte, the key is Ed25519, and the check returned false. Any uncertainty (an unparseable header, a component it can't rebuild, a non-Ed25519 algorithm, a runtime without Ed25519) is a warn, "could not verify". A correctly-signed directory can never be failed.

signature_freshness runs only after a signature verifies. A valid-but-expired signature is a warn, not a fail: it isn't forged, but a verifier that enforces expiry will likely reject it.

Two things to know before you ship

• Escape Finding.detail before rendering it as HTML. It's safe as a text node, but some strings echo values from the directory (a kid, a served path). Escape it if you build raw HTML.
• A pass means self-consistent, not authentic. A verified signature proves the response is cryptographically consistent with the directory's own advertised key. It does not prove the bot is who it claims. That check happens at request time, against a directory the verifier independently trusts.

Versioning

This release targets draft-meunier-http-message-signatures-directory-05 and RFC 9421. Drafts move. When a new draft changes behavior, check ids can be added, renamed, or regraded. While the package is pre-1.0, treat check ids as semi-stable: new ids may land in a minor release, and any rename or regrade will be called out in the changelog. If you gate CI on specific check ids, pin an exact version.

In production

Runs in production behind Sitedex's hosted checker at sitedex.dev/tools/keycheck. Same code, same findings.

Contributing

A new check is one add(check, grade, detail, authority, specRef?) call in validateDirectory (src/index.ts), or a SigFinding in src/signature.ts for a signature check. Two rules the test suite enforces:

• Every check id needs a CHECK_LABELS entry in src/refs.ts, or a label-coverage test fails CI.
• Every finding must declare an authority. If the rule is a real MUST/SHOULD, cite it with a specRef token that resolves through specRefUrl(). If it's Cloudflare's behavior or your own advice, say so; don't dress advice up as a standard.

License

Apache-2.0 © Sitedex