Signed snippets, hiccup, dual-eval, and durable path-addressed state.
Benchmark evaluation for tabular ML: Elo, win-rate, rank and improvability leaderboards (the engine behind TabArena)
Evaluate AI before you ship, while it runs and as you train it: agents, RAG pipelines, LLMs and models.
A collection of string-based validation functions, suitable for use in other Python 2 and 3 applications.
A pre-flight check for LLM eval results: catches partial runs, failed judges, mislabelled scores and inconsistent aggregation.
Builtin evaluators for agent-control
Galileo evaluators (Luna, Decision Models, LLM-as-judge) for agent-control
Mistral AI Studio evaluation tools
Shared utilities for METR Inspect AI eval repos: task scaffolder + common runtime helpers.
Package for fast computation of BSS Eval metrics for source separation
NeMo Evaluator — benchmark environments, pluggable solvers, interceptor proxy, and decision-grade scoring for LLMs
Python RPC client, official optimizer bridge, and DSPy metric adapter for @tangle-network/agent-eval.
Reproducible Subjective Evaluation
C++ OpenSCAD evaluator with Python bindings
Prime Intellect Evals SDK - Push and manage evaluations
Launcher for the evaluations provided by NeMo Evaluator containers with different runtime backends
PH Evaluator - an efficient Poker Hand Evaluator based on a Perfect Hash algorithm
A standalone, framework-agnostic evaluation and scoring engine for LLM/agent outputs — extracted from AgenticLens's evaluation module.
Discover and retrieve water data from U.S. federal hydrologic web services.
Evaluation tools for the SIGSEP MUS database
MCP server binary for eval-hub
Every Eval Ever
Evaluate AI-generated code, eval datasets and the evals themselves.
Agent evaluation toolkit
Evaluation of deep learning models
Regression gate for LLM apps: golden datasets, code + LLM-judge scoring, per-case baseline gating in CI, local run history.
Code for evaluating object tracking
Backwards-compatibility package for API of trulens_eval<1.0.0 using API of trulens-*>=1.0.0.
A library for evaluating LLM-based applications.
Evaluation plugin for Mistral Workflows - run evaluations as parallel Temporal activities
OmniAgent 开发者评测客户端
Python authoring SDK for reeval (*.eval.py eval files)
Stub package that mitigates Dependency Confusion attacks
Reusable evaluation contracts for binary classification: metrics, bootstrap CIs, calibration, artifacts, and evidence gates.
Audit an LLM eval set the way you would audit an exam: what it can measure, what is dead weight, and which items are broken.
Structured object comparison and evaluation library
Statistical gate for LLM eval changes: paired bootstrap CIs, blocking verdicts, tamper-resistant evidence
Evaluate, benchmark, and A/B-test AI coding agents (Claude Code, Codex, Gemini/Antigravity, OpenCode, Pi, Delegate) with sandboxed, reproducible YAML task suites.
Server binary for eval-hub
A Difference Evaluator for Alternating Images
Finds and removes malicious eval base64 PHP code.
EM algorithms for integrated spatial and spectral models.
Where does the noise in an eval score come from? Split it across questions, judges, prompt wordings and runs, and plan where to spend the next API calls
A small, plugin-based eval harness