scority-core

Scority

A ship / hold / rollback gate for automated changes — one that refuses to declare a win.

You changed something. Did it help? Most tooling answers that by comparing two numbers and calling the difference an effect. This does not:

pip install scority-core

from scority_engine.honesty import evaluate

# (before, after) per unit — pages, accounts, anything you can pair
result = evaluate(test_pairs, control_pairs=control_pairs)

result.decision.action       # 'expand' | 'hold' | 'rollback'
result.decision.reason       # 'CI straddles zero — effect not distinguishable from zero'
result.decision.confidence   # 'high' | 'medium' | 'low' — from the number of pairs
result.ci                    # CI(point, low, high, n, alpha) — difference-in-differences,
                             # 2000-iteration bootstrap, seeded

Four things have to be true before it will say expand: no watchdog regression, enough paired units (min_pairs, default 8), a confidence interval that does not cross zero, and a viable control cohort. Anything else is hold or rollback. There is no code path that turns "the numbers moved" into "we caused it".

That import costs nothing. scority_engine.honesty re-exports the decision, the sample-size rules, the contested-concept firewall and the fail-closed policy check — and loads no third-party package at all, not even the HTML parser the rest of the project uses. measure.py and holdout.py are 210 lines of standard library; the holdout split is sha1(salt|url), so the counterfactual is reproducible without carrying RNG state. tests/test_honesty_facade.py and tests/test_base_install_surface.py hold that line — the import chain used to drag in bs4 and pydantic, and nothing noticed until it was measured.

The reference implementation: SEO and AI visibility

The gate is not a thought experiment here — it runs against a real workload. Scority audits a page or a whole site, proposes concrete fixes, applies them to a holdout cohort, and puts the result through the decision above. The SEO and answer-engine modules are the proof that the discipline survives contact with a domain where wishful measurement is the norm, and they are useful on their own if that is what you came for.

The same refusal runs through the rest of it. Contested claims of the field — site authority as a standing factor, the "sandbox", CTR as a direct ranking factor — cannot be labelled verified no matter what produced them. A rate computed from a thin sample is contested, not a fact. See The rules, and where they live in the code.

Agent-facing contracts — what an agent may be, what one unit of work may ask of it, when a cached model answer may be reused — live in scority_engine/contracts/ with their own README.

Everything below is in this repository and runs offline unless a section says otherwise. Source paths are written relative to scority_engine/modules/ — audit/safe_fix/measure.py is scority_engine/modules/audit/safe_fix/measure.py on disk.

What this is not

• Not a rank tracker or a SERP data provider. serp/ is an adapter layer. You bring a provider key (Google via DataForSEO or Serper) or you use the modules that need no SERP at all.
• Not an autopilot. The open core stops at proposals, gates, and verdicts. The default policy decision for anything unmatched is REQUIRE_APPROVAL, and OUTREACH_SEND / FIX_CONFIRM can never be auto-allowed — an ALLOW rule targeting them is downgraded and the downgrade is written into the reason (policy/engine.py). Scheduling, delivery to production, and the human queue UI are not here.
• Not a traffic guarantee. There is no ROI projector and no "expected lift" estimator. predictive_ranking.py is a gate, not a predictor: it refuses to let a caller present a prediction until the evidence base clears explicit thresholds (200 observations, 60 days, 20 query/page pairs, 1000 impressions), and its declared policy string is literally hypothesis_ranker_with_ci_not_oracle. It has no callers in this repository, and that is the honest reading of it: the model it guards does not exist here, so the module is a contract for work that has not been written rather than machinery you can watch fire. Wiring it to the rule-based opportunity scorer would make it look active while gating nothing, which is worse than leaving it plainly inert.
• Not calibrated on a public gold-set. Weights and thresholds in the scoring modules are directional defaults. They are tagged # CALIBRATE in the source and versioned (THRESHOLDS_VERSION = "cg-v2", READINESS_VERSION, VV_VERSION) so a recalibration is a visible, diffable event rather than a silent one.
• Not a replacement for Search Console or an analytics suite. It reads them.
• Not fully offline. The technical audit, schema audit, citability, compliance scan, linking, llms.txt generation and the measurement math need no accounts and no keys. SERP, GSC, GA4 and AI-engine probes need credentials that are yours.

Quick start (60 seconds)

Python 3.12+.

git clone https://github.com/vitaliyino/scority-core.git
cd scority-core
uv venv && uv pip install -e ".[dev]"

The base install is nine packages. The technical audit, schema audit, citability, the compliance scan, linking, llms.txt generation and the measurement math all work with it and need no accounts. Heavier surfaces are extras — [service] (HTTP), [db] (Postgres), [llm] (embeddings), [mcp], [browser], or [all] — and a module that needs one says which, instead of failing with a bare ModuleNotFoundError. tests/test_base_install_surface.py holds that line: it imports each keyless module in a subprocess and fails if an optional dependency appears.

Check what the install can actually do before you rely on it. doctor lists every capability as OK or NO_CREDENTIALS — it never reports an unconfigured connector as an empty result, and it names any SCORITY_/PUBLIC_ variable you have set that nothing reads:

uv run scority doctor

One page, no accounts, no keys, no network beyond the page itself:

from scority_engine.modules.audit import audit_url

res = audit_url("https://example.com/")   # swap in your own page
print(res["score"], res["counts_by_severity"])
for f in res["findings"]:
    print(f["severity"], f["code"], f["title"])

69 {'MEDIUM': 3, 'LOW': 5}
MEDIUM meta_desc_missing Missing meta description
MEDIUM schema_missing No structured data (JSON-LD)
MEDIUM thin_content Thin content (21 words)
LOW canonical_missing No canonical tag
...

The whole site — crawl, technical audit sample, internal linking, UX/a11y, structured data, trust signals, local SEO and citability in one command:

uv run scority site-review https://example.com --pages 50 --markdown

Did last month's change actually work? Two CSV exports from Search Console — no account, no key, no API — and a verdict with an interval:

uv run scority rollout --before before.csv --after after.csv --site example.com \
  --test-url https://example.com/guides/p1 --test-url https://example.com/guides/p2

# Rollout measurement: example.com

**EXPAND** — CI entirely above zero — measured lift

- metric: `clicks` (higher is better)
- method: did
- pages paired: 14
- effect: +7.929 (95% CI +7.143 … +8.786)
- confidence: medium

--test-url names the pages that received the change; everything else becomes the control. Omit it and the cohort splits deterministically on sha1(salt|url), so the same export always yields the same split. What the command refuses to do is the point — see below.

Is this page structurally citable by AI answer engines?

uv run python -m scority_engine.modules.geo.readiness --url https://example.com/guide

site_review degrades section by section: if one analyzer throws, that section becomes a note in result["notes"] and the rest of the review still ships. A malformed page does not kill the run.

Give it to your agent (MCP)

Scority ships a Model Context Protocol server, so a coding agent can audit a page itself instead of you pasting URLs between windows. Bring your own keys — two of the four tools need none at all — seven of the eleven do.

uv tool install "scority-core[mcp]"    # or: pip install "scority-core[mcp]"

Claude Code:

claude mcp add scority -- scority-mcp

Cursor / Codex / anything else that speaks MCP — ~/.cursor/mcp.json or the client's equivalent:

{
  "mcpServers": {
    "scority": {
      "command": "scority-mcp",
      "env": {
        "SERPER_API_KEY": "optional — only for scority_serp",
        "DEEPSEEK_API_KEY": "optional — only for scority_content_gap"
      }
    }
  }
}

tool	what it does	what it costs
scority_audit(url)	on-page technical audit — fetch, parse, findings by severity	nothing; no account, no key
scority_ai_access(url)	which AI crawlers may read this site (robots.txt + llms.txt)	nothing; two requests
scority_llms_txt(url)	draft an llms.txt from the site's own sitemap	nothing; one sitemap fetch
scority_schema(url)	JSON-LD validity and rich-result correctness	nothing; one fetch
scority_site_review(url, max_pages)	crawl and review a whole site, summarised	nothing; max_pages requests
scority_compliance(text, niche)	statutory guards over a piece of text	nothing; pure and offline
scority_doctor()	what this install can do, and what needs a credential	nothing; reads config
scority_kb(query)	hybrid search over the SEO/GEO methodology corpus	nothing without embeddings configured
scority_serp(query, region)	top organic results from Google	one SERP API call
scority_content_gap(url, seed)	what competitors cover and this page does not	~$0.05 per page
scority_geo_wave(domain, out_dir)	do the AI engines cite this site, and did that change	one short completion per query per engine

A missing credential is a status, never an empty result. scority_serp without a provider key returns NO_CREDENTIALS naming the variables that would fix it — not {"results": []}, which an agent reads as "nobody ranks for this query". The same rule applies to an unreachable page: scority_audit returns UNREACHABLE with no score, because a 404 audits perfectly cleanly and a healthy number for a dead URL is the lie this engine exists to refuse.

The paid tool refuses by default. scority_content_gap returns a refusal naming the estimated cost unless you pass confirm=true. An agent cannot spend your money by inferring that you probably wanted the analysis — it has to be told, in the call, that spending is intended.

Every tool is read-only. Nothing in this server writes to a site, a repository or a database; the worst a confused agent can do is fetch a page you did not ask about.

Without the [mcp] extra the module raises a named error telling you to install it, rather than a bare ModuleNotFoundError from three frames down.

Working in Claude Code? skills/scority/SKILL.md is a ready skill — copy it into .claude/skills/scority/ and the agent knows which tool answers which question, which one costs money, and that it must not confirm the spend on your behalf.

Audit your pages in a pull request

A GitHub Action that runs the keyless audit and writes the result into the job summary. No account, no API key, no service to sign up for — it installs the package and fetches the pages.

# .github/workflows/seo.yml
name: SEO
on: [pull_request]

jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: vitaliyino/scority-core@v0.1.1
        with:
          urls: |
            https://example.com/
            https://example.com/pricing
          fail-on: none          # none | LOW | MEDIUM | HIGH | CRITICAL

fail-on: none reports without blocking. Watch a few pull requests, see which findings your team actually acts on, then tighten to MEDIUM or HIGH once the number means something. A threshold adopted before anyone has looked at the output is a threshold people learn to skip.

A page that does not answer fails regardless of the threshold. A 404 audits perfectly cleanly — an error page has no thin content and no missing canonical worth reporting — and reporting that as a clean result is precisely the dishonesty the rest of this project exists to refuse.

Pin the version. latest moves under you, and a check whose meaning changes without a commit is not a check.

What a verdict looks like

This is the honesty layer, and it is a pure function — deterministic, seeded, no network, no database. Paste this and you get exactly these numbers.

from scority_engine.modules.audit.safe_fix.measure import evaluate

# (before, after) CTR per page. test = got the fix, control = deliberately left alone.
test = [(0.021, 0.028), (0.019, 0.024), (0.031, 0.030), (0.012, 0.019), (0.026, 0.033),
        (0.017, 0.022), (0.023, 0.029), (0.014, 0.018), (0.028, 0.031), (0.020, 0.027)]
control = [(0.022, 0.023), (0.018, 0.017), (0.030, 0.031), (0.013, 0.014),
           (0.027, 0.026), (0.015, 0.016), (0.021, 0.022), (0.019, 0.018)]

print(evaluate(test, control_pairs=control).as_dict())

{
  "ci": { "point": 0.00475, "low": 0.00305, "high": 0.00625, "n": 10, "alpha": 0.05 },
  "method": "did",
  "decision": {
    "action": "expand",
    "reason": "CI entirely above zero — measured lift",
    "confidence": "low",
    "reason_code": "ci_above_zero"
  },
  "notes": []
}

Now drop the control cohort. Same test pages, same lift, different honesty:

{
  "ci": { "point": 0.005, "low": 0.0033, "high": 0.0063, "n": 10, "alpha": 0.05 },
  "method": "paired",
  "decision": { "action": "expand", "reason": "CI entirely above zero — measured lift", ... },
  "notes": ["no control cohort — paired estimate cannot separate the fix from a site-wide shift"]
}

And a change that did nothing — same control cohort, test pages that barely moved:

flat = [(0.021, 0.0215), (0.019, 0.0185), (0.031, 0.0312), (0.012, 0.0119), (0.026, 0.0261),
        (0.017, 0.0172), (0.023, 0.0228), (0.014, 0.0141), (0.028, 0.0279), (0.020, 0.0202)]

print(evaluate(flat, control_pairs=control).as_dict())

{
  "ci": { "point": -0.00021, "low": -0.00085, "high": 0.00051, "n": 10, "alpha": 0.05 },
  "method": "did",
  "decision": {
    "action": "hold",
    "reason": "effect not distinguishable from zero — keep gate",
    "confidence": "low",
    "reason_code": "ci_crosses_zero"
  },
  "notes": []
}

Note "confidence": "low" on a positive verdict. n=10 is below the module's own high bar (n≥30). The engine reports the lift and the thinness of the evidence in the same object.

What scority rollout refuses to do

The command above is this function fed from two Search Console exports, and the interesting part is what it drops on the way. A site is never the same set of pages in two windows: some are new, some stopped ranking. Both are trivially easy to score as a result of the change, and both would be a lie.

Run it on an export where one page vanished and one appeared:

"notes": [
  "control: 1 page(s) present only in the BEFORE window — excluded from the pairs, because scoring a disappearance as a drop to zero would attribute the loss of a page to the change",
  "control: 1 page(s) present only in the AFTER window — excluded from the pairs, because scoring an appearance as growth from zero would attribute a new page to the change"
]

Two more refusals sit behind it. Without a control cohort the method drops from did to paired and says in notes that it cannot separate the fix from a site-wide shift — it still answers, it just stops calling the answer causal. And below --min-pairs the verdict is hold with insufficient_data, never hold with "no effect": too little evidence and evidence of nothing are different findings, and a tool that conflates them teaches you to stop measuring.

The rules, and where they live in the code

Read the files. The claims below are one-line checks.

A verdict is computed, never asserted. audit/safe_fix/measure.py bootstraps the mean before→after delta (2000 iterations, seed=12345, alpha=0.05). With a control cohort it switches to difference-in-differences, so a site-wide algorithm shift that moves test and control equally nets out to approximately zero. The module docstring states plainly why it is not CausalImpact: real CausalImpact wants ~100 days and 40–50 pages, and a short window does not have that — so it reports "not distinguishable from zero" instead of claiming lift.

No signal, no expansion. evaluate() has exactly four exits before expand: any watchdog regression → rollback; n < min_pairs (default 8) → hold; CI entirely below zero → rollback; CI crossing zero → hold. expand requires ci.low > 0. loop.py adds one more: a cohort whose holdout is not viable is forced to hold — "autonomy never expands on a cohort it couldn't measure".

The counterfactual is real. audit/safe_fix/holdout.py splits a cohort by sha1(salt|url), so the same cohort always splits the same way and the split is reproducible without RNG state. The split is by page, never by user: one HTML per URL, no user-agent branching. viable is false unless both sides are non-empty and the cohort clears min_cohort.

Thin samples are never facts. provenance/confidence.py computes a Wilson score interval (asymmetric near 0 and 1, correct at the boundary) and derives trust status from sample size: n == 0 → unsupported, 0 < n < 30 → contested, n ≥ 30 → verified. proportion_provenance() is the one call a producer makes to ship a rate honestly — it will not stamp verified on a thin sample.

Contested SEO folklore cannot be laundered into fact. provenance/firewall.py holds a small versioned list of concepts that vendors and leaks present as ranking facts — site/domain authority as a standing factor, the "sandbox", CTR-and-dwell-time as a direct ranking factor, brand-mention causality. Any finding whose title, recommendation or evidence text matches is forced to status = "contested". A verified status cannot survive a match. Leak-derived heuristics ship as speculative from a tier-5 leak source, because a leak documents which attributes exist, not how ranking uses them (provenance/schema.py).

Provenance, and exactly how far it reaches. The canonical shape is {status, sources[], max_tier, ci, sample_size, method, prompt_version, contested_concept, measured_at}, with status ∈ {verified, contested, vendor-claim, speculative, unsupported} and method ∈ {self-measured, retrieval, coalition, probe}. It rides inside Finding.evidence["provenance"], so adding it broke no existing consumer.

This README used to say every number carries it. That was not true. Counting by the rule stated in tests/test_provenance_coverage.py — a module emits findings iff it constructs a Finding, or a dict literal carrying severity + title + recommendation that reaches a user under a findings key — 31 modules emit findings and 23 attach provenance. The other 8 attach nothing. The list is in that test file, and a new emitter fails the suite until it is classified, so this paragraph cannot quietly drift again. Closing one entry is a well-scoped contribution; they carry the provenance-gap label.

Everything the technical audit produces is stamped by construction (Finding.as_dict), as are the GEO access checks and the content gates. The unstamped set is mostly the analysers that grew before the schema existed.

The status is about the evidence, not about the advice. A finding whose evidence is a reading of the artefact — this JSON-LD block does not parse, this page has one inbound link, the phone in the markup differs from the phone in the text — is verified, and stays verified even when the threshold that makes it worth reporting was picked by eye; that threshold is disclosed separately as a # CALIBRATE marker. A finding whose evidence contains something nobody here measured is speculative: that a page is a "money page" is an inference about intent, and that a map embed is a trust signal is a belief about ranking. Both may be right. Neither was observed, and labelling them verified alongside a phone-number mismatch would empty the word out. Each module keeps that split in a _PROVENANCE table listing only its exceptions, so the decision is visible in the diff.

The causal claim waits for the window to close. outcome/journal.py records an intervention at apply time with a measure_after timestamp (default 28 days — one Search-Console-comparable window) and status PENDING. The delayed pass closes it as MEASURED or INCONCLUSIVE. outcome/metrics.py recomputes CTR from click and impression totals rather than averaging per-row CTRs, weights position by impressions, and reports a metric that exists on only one side of the window as only_before / only_after instead of silently treating the missing side as zero.

Windows are fixed, not "recent". baseline/snapshot.py builds two adjacent 28-day UTC windows and ends the current window three days before the seed date, because Search Console data arrives late. as_of can be pinned for replay.

Autonomy fails closed. policy/engine.py matches flat conditions (cost, risk class, URL/domain substring, complaint rate, UTC hour window). A condition containing any key the engine does not recognize never matches — an unrecognized rule can only fall back to the human gate, never grant autonomy. No matching policy at all returns REQUIRE_APPROVAL with reason no matching policy — default human gate. An auto-ALLOW still writes an approval record, so "the policy did it" is as auditable as "a human did it".

Fixes are reversible and TOCTOU-guarded. audit/safe_fix/changeset.py compiles engine fixes into a structured, adapter-agnostic, reversible operation set. Anything the engine cannot model structurally degrades to manual_action with applyMode: "manual" — never guessed. edge/rewrite.py applies a ChangeSet to HTML: only auto changes apply, a before value that no longer matches live HTML is a stop (skip, do not clobber), application is idempotent, and unknown ops are skipped with a reason.

Verification refuses convenient answers. pr/verify.py treats rel=nofollow, rel=sponsored and rel=ugc as a failed placement, not a verified one — because sponsored and guest-post links are exactly what outreach buys, and counting them as equity-passing would be a reporting false positive against the person paying for the report.

AI visibility is a separate axis

Scority measures AI-answer visibility as its own dimension and never folds it into a single site health score. In audit/audit.py the GEO readiness result is returned as a separate geo_readiness field with its own geo_findings list; it does not move the technical score. The reason is in the source: citability and trust are high-bar composites that most technically clean pages score poorly on, so folding them in would put noise on every page.

That gap has a name in the code — clean_but_invisible: a page that passes the technical audit and is still structurally uncitable.

• geo/ai_access.py — does robots.txt let answer engines in? It distinguishes retrieval bots (OAI-SearchBot, ChatGPT-User, PerplexityBot, Google-Extended — blocking these genuinely removes you from live AI answers, so it drives severity) from training bots (GPTBot, CCBot, anthropic-ai, ClaudeBot, Applebot-Extended — blocking these is a common, deliberate opt-out and is reported as INFO, not HIGH). Most tools conflate the two and inflate the finding.
• geo/bot_traffic.py — what AI crawlers actually fetch, parsed from server access logs. Most AI agents fetch without executing JavaScript, so client-side analytics never see them; logs are the only ground truth.
• geo/answer_monitor.py — repeated waves of queries against the answer engines you configure. The engine decides whether a domain was cited and whether it was recommended (endorsement ≠ mention), by deterministic matching over the response text; the model is not trusted to score itself. Rates carry Wilson intervals, and wave-over-wave change is a paired bootstrap on the difference in citation rates — an interval covering zero is stable, not "improving".
• citability/score.py — an offline 0–100 scorer for "would an answer engine lift this?" (answer-first block, structure, question headings, first-hand signals, heading quality). No keys, run it on your own drafts before publishing.
• llms_txt/generate.py — spec-valid llms.txt from a sitemap, stdlib-only, output round-trips through the validator. The docstring says out loud that Google states this is not a ranking signal; treat it as cheap crawler inventory.
• edge/agent_markdown.py — clean Markdown for AI crawlers, with an explicit guard: classic search crawlers are excluded from the AI user-agent list, because serving Googlebot a different stripped page would be cloaking — the exact harm this project exists to prevent.

Compliance packs

Generated text passes a deterministic guard before it can be published. compliance/us_pack.py ships the FTC pack: forbidden advertising claims tied to the rule they violate — guaranteed approval, "no credit check", instant money, guaranteed returns and debt relief, promises aimed at borrowers with poor credit, "100% success". Each pattern carries a block or verify severity and a message naming the basis (FTC Act §5 / §12).

Two details that matter more than the pattern list:

• A negation guard. "we do not guarantee approval" is a disclaimer, not a violation. Negation is clause-scoped, so a distant "not" in another sentence cannot suppress a real claim.
• Invented numbers are flagged separately. flag_unverifiable_specifics catches the figures a generator tends to fabricate about a specific brand — approval rate, client count, star rating, years in business, decision time — and marks them verify rather than blocking them. Grounding is value-aware: a number clears the flag only if it matches a fact you supplied, so a wrong figure still flags (compliance/facts.py).

The pack is a rule set, not a hardcoded jurisdiction. A second locale is a second pattern table plus its statute references. A Chinese pack (广告法 art. 9 superlatives as a hard block rather than a provable claim, art. 25 risk disclaimers, art. 28 unverifiable-evidence claims) is designed but not in this repository yet — it is on the roadmap, not in the code.

What is in the box

Area	Modules	Runs with no accounts
Technical audit	audit/ (checks, scoring, fix generation), schema_audit/, render/	yes
Whole-site review	site_review/, inventory/ (crawl + prioritized backlog), reporting/	yes
Measurement honesty	audit/safe_fix/ (holdout, measure, loop, changeset, preflight gates), baseline/, outcome/, provenance/	yes (math is pure)
AI visibility (GEO)	geo/, citability/, llms_txt/, edge/	mostly — engine probes need your keys
Content quality gates	content_gap/ (deterministic gates, YMYL thresholds, contested firewall), compliance/, knowledge/, text_blender/, page_gen/	gates yes; generation needs a model endpoint
Search data connectors	gsc/ (Great-Decoupling detector), ga4/, serp/, indexnow/	no — your own credentials
On-page quality	linking/ (PageRank-based internal linking), uxui/, local_seo/, eeat/, keyword_cluster/, migration_audit/, brand/	yes
Rendered-page checks	visual/, design_review/	needs a headless browser — playwright install chromium after the install above
Off-page	pr/ (donor scoring, honest backlink verification)	yes, read-only fetch
Autonomy	policy/ (fail-closed policy engine with a default human gate)	yes

schema_audit/ deserves a specific note: it catches the defect class that every JSON-LD flattener hides — blocks that fail json.loads and are silently skipped, so a page "has schema" that no engine can read. It also catches duplicate single-per-page types and a missing @context, on top of per-type validators for FAQPage, Product, BreadcrumbList, HowTo, Review, VideoObject, Event and JobPosting.

render/ matters for the same reason: fetch_or_render tries a cheap static fetch first and escalates to Chromium only when the response looks like a bot interstitial or a client-rendered shell. Without it, an SPA or a site behind bot protection produces a page full of confident false defects. The audit detects that case explicitly and returns blocked_or_js_shell with score: None instead of reporting "missing title".

Modules that document themselves

Eight of thirty-six. Each states what the module does, where it over- or under-matches, and what it cannot know — the last being the part most module docs leave out.

citability/	how easily an answer engine can lift and cite a page, and why its thresholds are directional secondary-source numbers rather than verified ones — contributed, and the model the rest follow
audit/safe_fix/	the expand / hold / rollback decision, with its own false-signal rate measured: at the default sample size a confident verdict on pure noise comes up about one run in eight
provenance/	the five statuses, the sample size below which a rate is not a fact, and the contested-concept firewall no verified survives
compliance/	the claim gate, how it tells a promise from a denial of the same promise, and every place it over-matches on purpose
geo/	what blocking each AI crawler actually does, sourced per row with the date the vendor page was read
gsc/	the Search Console measurement traps, starting with the row cap that costs 3 % of the impressions and 64 % of the pages
llms_txt/	the generator, and the question the genre argues about instead of measuring: of twelve vendor crawler docs, none says its crawler reads the file
contracts/	what an agent may be, what one unit of work may ask of it, and when a human has to sign

Writing these found defects rather than describing them: a comment that was wrong about the code it sat on, a parity test that only checked one direction, a guard narrower than the pattern it guarded. Documenting a module turns out to be the cheapest audit available, which is the argument for the remaining thirty.

Architecture

fetch → (escalate to render if stub) → parse → checks ─→ findings + provenance
                                                          │
                                                          ├─→ fixes → ChangeSet → preflight gates
                                                          │             (reversible, TOCTOU-guarded)
baseline windows ─→ holdout split (test / control) ─→ apply ─→ measure (paired | DiD)
                                                          │
                                            expand | hold | rollback
                                                          │
                                            intervention journal → delayed re-measure (28d)

Two invariants hold the whole thing together. The engine owns data, determinism and arithmetic; a language model, where one is used at all, owns wording and judgment and never computes its own verdict. And nothing becomes a fact without a sample size behind it.

The optional judgment layer (fable/) talks to any OpenAI-compatible chat endpoint you configure via base_url + API key. Its gates are composed by the engine, not by the model: ragas_gate passes a draft only if it is both faithful to its sources and relevant to the query — an AND over two separately thresholded scores, with the failing reason returned. Nothing in the audit, scoring, measurement or compliance path requires a model to be configured.

Open core vs hosted

Same split as Supabase: the engine is the product, the hosting is a convenience.

In this repository	In the hosted service
Audit, crawl, scoring, findings, fixes	Scheduled runs and the nightly loop
Holdout, measurement, verdicts, rollback paths	Publication gates, kill switch, budget enforcement
Provenance, contested firewall, compliance packs	Client portal, reports, billing
Policy engine (fail-closed, human gate by default)	Approval queue UI and the operator workflow
Connectors you configure with your own keys	Managed connectors and token custody
CLI and library API	Multi-tenant orchestration

You can run every capability in the left column yourself, forever, without an account. Nothing in the core phones home, and no module requires a Scority-hosted service to function.

Locales and markets

The package ships in one language, English, and one market. That is a narrowing, and it is worth being plain about what it cost: the vocabularies the scorers match against — the YMYL classifier, the citability scorer, the first-hand-signal detector — used to carry two pattern sets, and a second market is now a second pattern table rather than a flag.

• Reports render in English. There is no --lang; a parameter with one legal value is worse than none, because it invites a caller to pass the other one.
• Search data: Google Search Console and Google Analytics 4. GSC says whether people saw and clicked; GA4 says what the click became, which is the difference between "traffic went up" and "we got more enquiries". Both feed the same measurement core — there is one place where a number becomes a verdict. The rollout measurement (scority rollout) runs on the free GSC CSV export, so that verdict is available without an OAuth account at all.
• SERP: Google, via DataForSEO or Serper. The adapter interface is small enough to add another.
• IndexNow pings the participating engines after a fix, to shorten the gap between applying a change and being able to measure it. There is deliberately no Google ping: Google does not support IndexNow and retired its sitemap-ping endpoint in 2023, so shipping one would be cargo cult.
• Compliance: the FTC pack ships. A Chinese pack is designed and not yet implemented (see above), and the issue template for contributing another is locale_pack.yml.

Status

Extracted from a production system that has been running against live client sites. This repository has no history from that system by design — it starts clean.

The measurement, provenance, policy and compliance layers are pure functions with offline tests and no fixtures. Scoring weights are directional and tagged for calibration; when we calibrate, the version string changes and you can diff it. If you find a claim in this README that the code does not hold, that is a bug and we want the issue.

Contributing

House rules, in order of how much we care:

• A scorer is a pure function. No network, no database, no clock inside anything that produces a score. If it needs data, it takes data as an argument.
• No new verified without a sample-size gate. If you emit a rate, emit it through provenance.proportion_provenance or explain in the PR why the existing gate does not apply.
• Thresholds are tagged and versioned. New weights get # CALIBRATE and bump the module's version constant.
• Fail closed. An unrecognized rule, an unmodellable fix, a cohort too small to measure — each falls back to the human gate. Never to autonomy.
• Tests run offline. No fixtures fetched at test time, no live endpoints.
• Don't add a claim the code cannot hold, in the code or in the docs.

Issues and pull requests welcome. If you are adding a locale pack, open an issue first with the statute references — the citation is the hard part, the regex is not.

License

Apache-2.0. See LICENSE.