New:Socket for Asana Is Now Available.Learn more →
Sign In

@ultimat3/testing

Package Overview
Dependencies
Maintainers
1
Versions
23
Alerts
File Explorer

Advanced tools

License
Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

@ultimat3/testing - npm Package Compare versions

Comparing version
9.0.0
 to
10.0.0
+2
-1
CLAUDE.md

@@ -48,3 +48,4 @@ # @ultimat3/testing — boundary

| The same is still true of permissions, and it is MEASURED, `As of 2026-08-23` | permissions, roles and the locale config are still replaced with the file's baseline, so an app's `definePermissions()` reached only by a dynamic `loadApp()` is dropped at that file's boundary exactly as the catalogs were. Reproduced: `bun test apps packages` in `dummy/social-media-clone` — one process, unsharded — leaves 6 `.contract.` cases failing on `knownPermissions()` missing `dashboard:read`, and every one of them passes when its file runs alone. The catalog fix took that run from 16 fail to 6; these are the 6. Not fixed here, and the reason is not that it is a different defect — it is the same one — but that the same union rule applied to permissions leaks every permission `packages/policy/src/permissions.test.ts` declares into every later file, and judging that needs a repo-wide `bun test` this package cannot run for itself. Its own piece of work, not a rider on this one. The `unit` step is green over it because `.contract.` is a different step and it shards |
| Guarded state is boot state | only the two registries whose honest invariant is "clean when the file ends" — `declareTags` and `registerTier` are boot installs. `entity()`, `job()` and `defineRoute()` register at MODULE scope, which is how an app declares itself, so a filled registry there is idiomatic and unguarded |
| Guarded state is boot state | only the two registries whose honest invariant is "clean when the file ends" — `declareTags` and `registerTier` are boot installs. `entity()`, `job()` and `defineRoute()` register at MODULE scope, which is how an app declares itself, so a filled registry there is idiomatic and unREPORTED |
| Filled and CLEARED are different questions | and the row above answers only the first. "Idiomatic to leave filled" is about the leak REPORT; it says nothing about a file that calls `clear*()`/`reset*()` and takes a module-scope declaration away from every file after it, which is what the RESTORE half exists for. The restore covers four registries and roughly nine have a reset export — routes (`clearRoutes`, `@ultimat3/render`), jobs and tasks (`@ultimat3/jobs`), actions and queries (`resetRegistry`), models/prompts/agents (`@ultimat3/ai`), mails (`@ultimat3/mail`). The table in `registry-leak-guard.ts` is the list. **Not closed, `As of 2026-08-23`**, and the reason is the shape rather than the difficulty: `jobs`, `action`, `query` and `mail` publish a lister and a reset but no RESTORE, so each needs the pair `@ultimat3/policy` got (`restorePermissions` / `restoreRoles`) before one line here can use it — and it has to land as one change, because anything less means editing `ProcessRegistrySnapshot` twice |
| An empty registry is a premise you state | a test whose subject is "nothing is declared" — `x db gen` with nothing to generate — calls `isolateEntityRegistry()` and restores in a `finally`. Inheriting it means the test passes until a neighbouring file imports an entity |

@@ -51,0 +52,0 @@ | That one helper is off the barrel | `@ultimat3/testing/registry-isolation`, its own entry point. It is the only module here that value-imports `@ultimat3/entity` — the restore is handed back synchronously, so it cannot be a dynamic import inside the call — and a static re-export from `src/index.ts` would load the entity registry into every test that imports this package for `expect` |

+12
-12
package.json
{
"name": "@ultimat3/testing",
"version": "9.0.0",
"version": "10.0.0",
"description": "Test harness: cloned template DBs per worker, frozen clock, sealed network, 6 test types",

@@ -36,14 +36,14 @@ "license": "MIT",

"dependencies": {
"@ultimat3/cache": "9.0.0",
"@ultimat3/core": "9.0.0",
"@ultimat3/db": "9.0.0",
"@ultimat3/entity": "9.0.0",
"@ultimat3/i18n": "9.0.0",
"@ultimat3/jobs": "9.0.0",
"@ultimat3/mail": "9.0.0",
"@ultimat3/policy": "9.0.0",
"@ultimat3/query": "9.0.0",
"@ultimat3/realtime": "9.0.0",
"@ultimat3/time": "9.0.0"
"@ultimat3/cache": "10.0.0",
"@ultimat3/core": "10.0.0",
"@ultimat3/db": "10.0.0",
"@ultimat3/entity": "10.0.0",
"@ultimat3/i18n": "10.0.0",
"@ultimat3/jobs": "10.0.0",
"@ultimat3/mail": "10.0.0",
"@ultimat3/policy": "10.0.0",
"@ultimat3/query": "10.0.0",
"@ultimat3/realtime": "10.0.0",
"@ultimat3/time": "10.0.0"
}
}
+6
-18
src/errors.ts

@@ -65,3 +65,8 @@ // The X_* codes owned by @ultimat3/testing. A test failure has to be as actionable as a runtime

const docsFor = (code: TestingErrorCode): string => `https://ultimate.dev/errors/${code}`;
// No `docs:` on the subclasses below. `UltimateError` fills it from `describeErrorCode(code).docs`,
// which is `@ultimat3/core`'s `ERROR_DOCS_URL` — one page for every code, never one per code, because
// `wiki/` is the framework's only public documentation surface and a code lives there in a TABLE ROW,
// which has no anchor. The `https://ultimate.dev/errors/<code>` links this file built until 9.x
// answered 404, host included, on every error it has ever thrown; restating the replacement here
// would be the same constant in eight places waiting to drift again.

@@ -77,3 +82,2 @@ /** A test reached the network without a mock or an allowlist entry. Always a bug, never a flake. */

fix: `mockFetch('${input.url}', () => new Response('{}')) — or allowHost('${hostOf(input.url)}') if it must be real`,
docs: docsFor('X_TEST_NETWORK_SEALED'),
});

@@ -90,3 +94,2 @@ }

fix: 'x dev (embedded Postgres), or set TEST_DATABASE_URL to a running Postgres',
docs: docsFor('X_TEST_DB_UNAVAILABLE'),
});

@@ -103,3 +106,2 @@ }

fix: 'wrap the test in frozenClock() / seededRandom(), or remove the wall-clock read',
docs: docsFor('X_TEST_NONDETERMINISTIC'),
});

@@ -131,3 +133,2 @@ }

fix: `register it at test setup: defineFixtures({ ${input.name}: () => buildIt() })`,
docs: docsFor('X_TEST_FIXTURE_UNKNOWN'),
});

@@ -153,3 +154,2 @@ }

fix: `install one in the test preload: defineFixtures({ ${input.name}: () => yourDriver() })`,
docs: docsFor('X_TEST_FIXTURE_UNAVAILABLE'),
});

@@ -179,3 +179,2 @@ }

fix: "import the app's api module in the test preload — import './apps/web/api' — then: x queries list --json",
docs: docsFor('X_TEST_LIVE_NODE_EMPTY'),
});

@@ -199,3 +198,2 @@ }

fix: 'await node.start() before connect(), and keep the request path at /_x/sync',
docs: docsFor('X_TEST_LIVE_NODE_UPGRADE_REFUSED'),
});

@@ -219,3 +217,2 @@ }

fix: 'network.online() before the call — or assert the offline path instead of the request',
docs: docsFor('X_TEST_NETWORK_OFFLINE'),
});

@@ -232,3 +229,2 @@ }

fix: 'improve the prompt under test, or lower the threshold passed to evalTest()',
docs: docsFor('X_TEST_EVAL_THRESHOLD'),
});

@@ -247,3 +243,2 @@ }

fix: 'call toRejectInput(action.input) — the schema, not toRejectInput(action) or the query',
docs: docsFor('X_TEST_SCHEMA_EXPECTED'),
});

@@ -261,3 +256,2 @@ }

fix: 'call toEmitSteps(myJob) with the job export, not toEmitSteps(myJob.run)',
docs: docsFor('X_TEST_JOB_EXPECTED'),
});

@@ -281,3 +275,2 @@ }

fix: `declare it: defineFactory(${input.table}, { traits: { ${input.trait}: { /* columns */ } } })`,
docs: docsFor('X_TEST_FACTORY_TRAIT_UNKNOWN'),
});

@@ -298,3 +291,2 @@ }

fix: 'usePersister({ insert: (table, row) => repoFor(table).insert(row) }) in the test preload — or build() for an in-memory row',
docs: docsFor('X_TEST_FACTORY_NOT_PERSISTED'),
});

@@ -315,3 +307,2 @@ }

fix: 'do not call unsealNetwork() while a request from the same test is still in flight',
docs: docsFor('X_TEST_NETWORK_RACE'),
});

@@ -375,3 +366,2 @@ }

fix: `${input.leaks.map(repairFor).join('; ')} — then re-run: bun test ${files.join(' ')}`,
docs: docsFor('X_TEST_REGISTRY_LEAK'),
});

@@ -403,3 +393,2 @@ }

: `mountIsland({ build, root, file: ${renderFixLiteral(input.built[0], ISLAND_PLACEHOLDER)} }) — the path is app-root-relative, not relative to the test, and ${file} is not one of them`,
docs: docsFor('X_TEST_ISLAND_NOT_BUILT'),
});

@@ -454,3 +443,2 @@ }

: `in ${file} add: import { render } from 'solid-js/web'; export function mount(el: HTMLElement, props: Parameters<typeof ${component}>[0]): void { el.textContent = ''; render(() => <${component} {...props} />, el); }`,
docs: docsFor('X_TEST_ISLAND_NO_MOUNT'),
});

@@ -457,0 +445,0 @@ }

+32
-4
src/registry-leak-guard.ts

@@ -17,9 +17,34 @@ // Cross-file state pollution, caught at the boundary it crosses and — where a registry can be put

* are not here: `entity()` and `job()` register at module scope, which is how an app declares
* itself, so a file that leaves them filled is idiomatic rather than leaky. A test whose subject is
* itself, so a file that leaves them FILLED is idiomatic rather than leaky. A test whose subject is
* an EMPTY one of those establishes it itself — `isolateEntityRegistry()`.
*
* Neither is RESTORED, and that is the same judgement read the other way: `@ultimat3/cache`
* publishes no un-declare for a tag, so there is nothing to put a tag registry back WITH. The
* Neither is RESTORED, and for the tag registry that is the same judgement read the other way:
* `@ultimat3/cache` publishes no un-declare, so there is nothing to put one back WITH. The
* registries that are restored are `registry-snapshot.ts`'s, and none of them is reported —
* repairing a state and then failing the run over it would be two answers to one question.
*
* **Filled and CLEARED are different questions, and the paragraph above only answers the first.**
* "Idiomatic to leave filled" says nothing about a file that calls `clear*()`/`reset*()` and takes
* a module-scope declaration from every file after it — a module evaluates once per process, so the
* next file's own `import` is a cache hit that declares nothing. That is what `registry-snapshot.ts`
* repairs, and it repairs FOUR registries out of the set that has one:
*
* | registry | reset export | owner | in the snapshot? |
* |---|---|---|---|
* | locales / catalogs | `resetCatalogs` | `@ultimat3/i18n` | yes |
* | permissions / roles | `restorePermissions` / `restoreRoles` | `@ultimat3/policy` | yes |
* | routes | `clearRoutes` | `@ultimat3/render` | **no** |
* | jobs | `resetJobs` | `@ultimat3/jobs` | **no** |
* | tasks | `resetTasks` | `@ultimat3/jobs` | **no** |
* | actions | `resetRegistry` | `@ultimat3/action` | **no** |
* | queries | `resetRegistry` | `@ultimat3/query` | **no** |
* | models / prompts / agents | `resetModels` / `resetPrompts` / `resetAgents` | `@ultimat3/ai` | **no** |
* | mails | `resetMails` | `@ultimat3/mail` | **no** |
* | entities | `clearRegistry` | `@ultimat3/entity` | no, by decision — `registry-isolation.ts` |
*
* Every "no" needs the SAME two halves the two "yes" rows have: a reader and a writer in the owning
* package (`restorePermissions` was added to `@ultimat3/policy` for exactly this), and one line
* here. `@ultimat3/jobs`, `@ultimat3/action`, `@ultimat3/query` and `@ultimat3/mail` publish a
* lister and a reset but no restore, so the change is theirs first and this file's second — and it
* has to land as ONE change, because the cost of the rest is a second edit of this same shape.
*/

@@ -142,3 +167,6 @@ export interface RegistrySample {

// `.tsx` through `transformTsx` is not the alternative either: it needs `@ultimat3/render`,
// whose import installs that global loader into every test process in the repo. Zero
// and the transform itself is on `@ultimat3/render/server`, whose import installs that global
// loader into every test process in the repo. (The `.` barrel does NOT — `installRenderLoader()`
// moved to `server.ts:15` in the 9.0.0 split, and this comment claimed otherwise until
// 2026-08-23. The `server` half is still the one that would have to be imported.) Zero
// `.test.tsx` files exist and the convention is `<file>.test.ts`, so the narrower filter

@@ -145,0 +173,0 @@ // costs nothing today; a `.test.tsx` added later is unguarded rather than mis-compiled.

+5
-0
src/registry-snapshot.ts

@@ -5,2 +5,7 @@ // The process registries a test file inherits, captured and handed back at the file boundary;

// permanent and there is no second evaluation left to redo it.
//
// FOUR registries, out of roughly nine that publish a `clear*`/`reset*`. The missing ones and what
// each still needs are tabulated in `registry-leak-guard.ts` beside `RegistrySample` — every one of
// them needs a RESTORE in its owning package first, the way `restorePermissions` / `restoreRoles`
// were added to `@ultimat3/policy` for the two rows below.

@@ -7,0 +12,0 @@ import type { Catalog, Locale, LocaleConfig } from '@ultimat3/i18n';