Benchmarking

Simple benchmark framework (in active development)

A ``pytest`` fixture for benchmarking code. It will group the tests into rounds that are calibrated to the chosen timer.

Pytest plugin to create CodSpeed benchmarks

provides a common interface to many IR ad-hoc ranking benchmarks, training datasets, etc.

OpenMMLab Detection Toolbox and Benchmark

Airspeed Velocity: A simple Python history benchmarking tool

Store data created during your pytest tests execution, and retrieve it at the end of the session, e.g. for applicative benchmarking purposes.

Reversible Data Transforms

Python module to run and analyze benchmarks

Metrics for multiple object tracker benchmarking.

Benchmarking QRC measures the ability to store information of

Massive Text Embedding Benchmark

The official SWE-bench package - a benchmark for evaluating LMs on software engineering

A Python Toolbox for Benchmarking Machine Learning on Partially-Observed Time Series

Official Implementation of "COLLIE: Systematic Construction of Constrained Text Generation Tasks"

Python Automatic Forecasting

Benchmark suite for Autoregressive Neural Emulators of PDEs in JAX.

Open MMLab Semantic Segmentation Toolbox and Benchmark

Benchmark Runner Tool

OpenMMLab Pose Estimation Toolbox and Benchmark.

resp-benchmark is a benchmark tool for testing databases that support the RESP protocol, such as Redis, Valkey, and Tair.

A Python wrapper for the Penn Machine Learning Benchmark data repository.

Merlion: A Machine Learning Framework for Time Series Intelligence

A Heterogeneous Benchmark for Information Retrieval

Open Graph Benchmark

WebArena benchmark for BrowserGym

OpenMMLab Image Classification Toolbox and Benchmark

Collection of ML models and benchmarking tools

MiniWoB++ benchmark for BrowserGym

Tools to benchmark, deploy and monitor prediction market agents.

Quick and easy python benchmarking.

Fuzzy Data Benchmark

Silero Models: pre-trained enterprise-grade STT / TTS models and benchmarks.

ML models + benchmark for tabular data classification and regression

OpenMMLab Model Pretraining Toolbox and Benchmark

A high-performant C++ implementation of benchmark functions for mathematical optimization algorithms.

WorkArena benchmark for BrowserGym

AssistantBench benchmark for BrowserGym

This is an unofficial, use-at-your-own risks port of the webarena benchmark, for use as a standalone library package.

VisualWebArena benchmark for BrowserGym

Modern benchmarking library for python with pytest integration.

Benchmark your code

BrowserGym integration for the WebLINX benchmark

A library to benchmark code snippets.

This is an unofficial, use-at-your-own risks port of the visualwebarena benchmark, for use as a standalone library package.

A public and reproducible collection of reference implementations and benchmark suite for distributed machine learning systems.

CLIP-like models benchmarks on various datasets

robosuite: A Modular Simulation Framework and Benchmark for Robot Learning

Scikit-learn-compatible datasets