Eval

Show value of an importable object

Like `typing._eval_type`, but lets older Python versions use newer typing features.

HuggingFace community-driven open-source library of evaluation

Safely evaluate AST nodes without side effects

Safe, minimalistic evaluator of python expression using ast module

A simple, safe single expression evaluator library.

Evalica, your favourite evaluation toolkit.

Validation and secure evaluation of untrusted python expressions

Testing framework for sequence labeling

The open-source evaluation framework for LLMs.

LLM Evaluations

an AutoML library that builds, optimizes, and evaluates machine learning pipelines using domain-specific objective functions

A getattr and setattr that works on nested objects, lists, dicts, and any combination thereof without resorting to eval

Python Mathematical Expression Evaluator

A framework for evaluating language models

Use EvalAI through command line interface

Evaluation tools for the SIGSEP MUS database

MS-COCO Caption Evaluation for Python 3

A poker hand evaluation and equity calculation library

EvalScope: Lightweight LLMs Evaluation Framework

Provides Python bindings for popular Information Retrieval measures implemented within trec_eval.

A library for providing a simple interface to create new metrics and an easy-to-use toolkit for metric computations and checkpointing.

evalutils helps users create extensions for grand-challenge.org

"EvalPlus for rigourous evaluation of LLM-synthesized code"

Limited evaluator

Lazy computing tree cache library

Evaluating and scoring financial data

Data Evaluation Software of the (Magnetism) research group of Prof. Ehresmann at University of Kassel

Faster interpretation of the original COCOEval

Universal library for evaluating AI models

Provides Python bindings for popular Information Retrieval measures implemented within trec_eval.

A powerful web content fetcher and processor

eval-mm is a tool for evaluating Multi-Modal Large Language Models.

A library for providing a simple interface to create new metrics and an easy-to-use toolkit for metric computations and checkpointing.

Contains the integration code of AzureML Evaluate with Mlflow.

User-friendly evaluation framework: Eval Suite & Benchmarks: UHGEval, HaluEval, HalluQA, etc.

A Difference Evaluator for Alternating Images

Backwards-compatibility package for API of trulens_eval<1.0.0 using API of trulens-*>=1.0.0.

A custom Streamlit component to evaluate arbitrary Javascript expressions.

Evaluation

A flexible, generalized tree-based data structure.

An information retrieval evaluation script based on the C/W/L framework that is TREC Compatible and provides a replacement for INST_EVAL, RBP_EVAL, TBG_EVAL, UMeasure and TREC_EVAL scripts. All measurements are reported in the same units making all metrics directly comparable.

Performs async evaluations of strings

AlpacaEval : An Automatic Evaluator of Instruction-following Models

Package for fast computation of BSS Eval metrics for source separation

Send Sir Perceval on a quest to fetch and gather data from software repositories.