The `aid` command: scaffold a project, validate its spec, generate its code, and run its eval gate.
Portable Tasksets, graders, evaluation runs, receipts, conformance fixtures, and Work evidence for OpenPond Harnesses
AID runtime for TypeScript: pinned-model gateway, prompt registry, tool dispatch, pipelines, structured output, eval gates and retrieval.
Evaluates a rational function.
Codegen for target: eval. Single-file agent.ts that boots dataset-registry + grader-registry + eval-runner.
Simulate a Recipe's bindings in evals: MCP and API fakes served from their captured contracts, and Google and Microsoft 365 emulators
OuterLayer CLI — commands: setup, init, claude, opencode, codex, connect, doctor, login, logout, sync, daemon (alias: watch), emit, demo, findings, work, run, spec, runner, hooks, mcp, context. Capture and sync coding-agent sessions, and record proof of f
eval-genius enables evals of arbitrary async code. It is generally intended for making multiple assertions on outputs which are generated nondeterministically. These assertions can be used to score algorithms on their effectiveness.
genroc script tasks for Node: the author-time bundler and the queue worker
Log-as-eval-fixture and LLM evaluation harness for Loom — replay a recorded event log deterministically, diff runs for golden-fixture regression, score outcomes, and run graded suites (exact, regex, JSON Schema, LLM judge via the decide scope) over casset
Scaffold the PMOS portable kit — run the PMOS product-management workflow (initiative → lane-fit spec → contract → build → three gates, with work-type lanes for features, research, maintenance, incidents, and content) entirely inside your own repo, with a
Skill execution telemetry & effectiveness scoring for DeepSeek Harness: records every skill load, folds the follow-up window into evidence, scores each skill and hands the agent concrete optimization advice.
Build agent applications with typed chat, tools, sandboxes, integrations, billing, and evaluation.
Workflow evaluations for CEF: datasets, experiments, scoring and comparison over an injected store and workflow target.
Eval runner — per-sample isolation, concurrency, grading, persistence
Shape Expressions triple expression evaluator api - defines how @shexjs/validator invokes regex evaluators.
Evaluate expressions from the SurveyJS standard.
Record every time AI writes wrong code and turn those failures into a project-specific eval suite. Local-only.
Headless exercise engine (shared types + grading + runners) for educspace tools (slate, stevenson templates).
Deterministic, rule-based regression testing for AI-powered HTTP endpoints.
TypeScript definitions for static-eval
Shape Expressions triple expression evaluator - simple regexp returning 1 error.
Shape Expressions triple expression evaluator - hard-working regexp engine returning multiple errors.
Test any AI model the same way. One plugin per vendor (Bedrock, OpenAI, Anthropic, Gemini, Ollama), code assertions by default, optional cheap-first LLM judge, per-model cost, and a CI release gate.
Evaluation tools for AI components, functions, workflows, and agents
Harness de evals para servidores MCP: seleção de tool em turno único (catálogo, fixtures, scorer, gate, runner via Messages API) e SESSÃO LONGA (loop client-side por stdio, A/B do bloco `retrieval`, métrica de chamadas ruins por tarefa, teto de gasto, NDJ
Create a lazily evaluated value
Deterministic reference scorers as AIP-14 TOOL contracts + one builtin AIP-30 PROVIDER. A scorer IS a tool whose outputSchema is the shared `Score` shape — no new scorer port. Ships exact-match, regex-match, json-schema-valid, and latency-budget. No LLM/m
Erdo CLI — drive datasets, pages, and evals from the terminal or CI
Mathematical expression evaluator
Regression eval gates for DSH agents: case definitions, baseline comparison and CI verdicts.
What a plugin eval's and the AI judge's server and runner both execute with, so the two never drift: the plugin-eval format's transcript reader, its expect dialect and a linear matcher, mock answer rendering, the wording of a try's own stops, and the AI j
Zero-config Node.js wrapper for M365 Copilot Agent Evaluations CLI (Python-based Azure AI Evaluation SDK)
Generates code from mathematical expressions
LLM evals for JavaScript and TypeScript apps, in one bash file: a worker helper and the tinyeval command.