Show value of an importable object
Like `typing._eval_type`, but lets older Python versions use newer typing features.
Safely evaluate AST nodes without side effects
HuggingFace community-driven open-source library of evaluation
A simple, safe single expression evaluator library.
Framework for evaluating stochastic code execution, especially code making use of LLMs
Safe, minimalistic evaluator of python expression using ast module
Universal library for evaluating AI models
The official Python library for the CoreWeave Evaluations API
The LLM Evaluation Framework
Open-source evaluators for LLM applications
"EvalPlus for rigourous evaluation of LLM-synthesized code"
A framework for evaluating language models
Collection of large language model evaluations
Provides Python bindings for popular Information Retrieval measures implemented within trec_eval.
EvalScope: Lightweight LLMs Evaluation Framework
Validation and secure evaluation of untrusted python expressions
LLM Evaluations
Self-hosted system for improving AI agents: server, CLI, web UI, Python SDK and agent skills
Open-source evaluators for LLM agents
A poker hand evaluation and equity calculation library
Testing framework for sequence labeling
An evaluation framework library for Python that provides a flexible way to run parallel evaluations and optionally integrate with the Orq AI platform.
Faster interpretation of the original COCOEval
Python SDK for EvalHub: common models, REST API client, and framework adapter SDK
Common metrics for common audio/music processing tasks.
One-click accuracy evaluation harness for SGLang.
Evaluation framework for Strands
The open source post-building layer for Agent Behavior Monitoring.
Python Mathematical Expression Evaluator
Evalica, your favourite evaluation toolkit.
A library for providing a simple interface to create new metrics and an easy-to-use toolkit for metric computations and checkpointing.
Verifiers: Environments for LLM Reinforcement Learning
The official Python SDK for Eval Protocol (EP.) EP is an open protocol that standardizes how developers author evals for large language model (LLM) applications.
Provides Python bindings for popular Information Retrieval measures implemented within trec_eval.
MS-COCO Caption Evaluation for Python 3
A getattr and setattr that works on nested objects, lists, dicts, and any combination thereof without resorting to eval
Subpackage for evaluation in NVIDIA NeMo Agent Toolkit
Microsoft Azure Evaluation Library for Python
Comprehensive AI Model Evaluation Framework with support for multiple LLM providers
Limited evaluator
Package for fast computation of BSS Eval metrics for source separation
Builtin evaluators for agent-control
Shared utilities for METR Inspect AI eval repos: task scaffolder + common runtime helpers.
Berkeley Function Calling Leaderboard (BFCL)
Galileo evaluators (Luna, Decision Models, LLM-as-judge) for agent-control
A custom Streamlit component to evaluate arbitrary Javascript expressions.