Show value of an importable object
Like `typing._eval_type`, but lets older Python versions use newer typing features.
Safely evaluate AST nodes without side effects
HuggingFace community-driven open-source library of evaluation
A simple, safe single expression evaluator library.
Framework for evaluating stochastic code execution, especially code making use of LLMs
Safe, minimalistic evaluator of python expression using ast module
Universal library for evaluating AI models
The official Python library for the CoreWeave Evaluations API
The LLM Evaluation Framework
Open-source evaluators for LLM applications
EvalScope: Lightweight LLMs Evaluation Framework
Validation and secure evaluation of untrusted python expressions
A framework for evaluating language models
A poker hand evaluation and equity calculation library
LLM Evaluations
Provides Python bindings for popular Information Retrieval measures implemented within trec_eval.
"EvalPlus for rigourous evaluation of LLM-synthesized code"
An evaluation framework library for Python that provides a flexible way to run parallel evaluations and optionally integrate with the Orq AI platform.
Collection of large language model evaluations
Open-source evaluators for LLM agents
Common metrics for common audio/music processing tasks.
Python Mathematical Expression Evaluator
Evaluation framework for Strands
The open source post-building layer for Agent Behavior Monitoring.
Testing framework for sequence labeling
The official Python SDK for Eval Protocol (EP.) EP is an open protocol that standardizes how developers author evals for large language model (LLM) applications.
Faster interpretation of the original COCOEval
A library for providing a simple interface to create new metrics and an easy-to-use toolkit for metric computations and checkpointing.
Provides Python bindings for popular Information Retrieval measures implemented within trec_eval.
EVALMY.AI Python library
Python SDK for EvalHub: common models, REST API client, and framework adapter SDK
MS-COCO Caption Evaluation for Python 3
One-click accuracy evaluation harness for SGLang.
A getattr and setattr that works on nested objects, lists, dicts, and any combination thereof without resorting to eval
Microsoft Azure Evaluation Library for Python
Verifiers: Environments for LLM Reinforcement Learning
Limited evaluator
An AWS Labs Model Context Protocol (MCP) server for Bedrock Knowledge Base Retrieval
Comprehensive AI Model Evaluation Framework with support for multiple LLM providers
Subpackage for evaluation in NVIDIA NeMo Agent Toolkit
Berkeley Function Calling Leaderboard (BFCL)
A custom Streamlit component to evaluate arbitrary Javascript expressions.
Evaluation Framework
Agent evaluations on seeded MCP worlds, recorded and compared in Langfuse.
Evaluation harness for Vision-Language-Action models