Benchmarking

Easily benchmark Machine Learning models on selected tasks and datasets - with PyTorch

Benchmark suite tool

A Python package to benchmark query performance and comparison on PostgreSQL Database

OGBench: Benchmarking Offline Goal-Conditioned RL

Energy forecast benchmarking toolkit.

evalsync is a library used to synchronize applications under benchmark with an external manager

ycecream - sweeter debugging and benchmarking

A simple ANN benchmark tools

Temporal Graph Benchmark project repo

I/O profiler for deep learning python apps. Specifically for dlio_benchmark.

High-level Interface to Inertial Motion Tracking

Rotation Detection Toolbox and Benchmark

A Python interface for the USEPA Benchmark dose modeling software (BMDS)

perfbench measures execution time of code snippets with Timeit and uses Plotly to visualize the results.

A comprehensive benchmark for real-world Sentinel-2 imagery super-resolution

A centralized benchmarking suite to facilitate comparative profiling of tools across graph analytic libraries and datasets

Benchmark functions for Bayesian optimization

Easily benchmark Machine Learning models on selected tasks and datasets

A pytest-like framework for benchmarking

Opinionated Benchmarking Automatation in Galaxy

A PyTorch library for benchmarking deep metric learning. It's powerful.

Easily benchmark PyTorch model FLOPs, latency, throughput, max allocated memory and energy consumption in one go.

Vision Document Retrieval (ViDoRe): Benchmark. Evaluation code for the ColPali paper.

Evolution Gym: A benchmark for developing and evaluating soft robot co-design algorithms.

Benchmarking framework for all types of black-box optimization algorithms, postprocessing.

Small Benchmarks for LM Agents

Platform Performance Benchmarking

RelBench: Relational Deep Learning Benchmark

Python Cache Hierarchy Simulator

ジョブショップスケジューリング問題のベンチマーク問題をpythonのクラスで表現し，load出来るようにする

Benchmark tabular synthetic data generators using a variety of datasets

Minimal Python library to connect to LLMs (OpenAI, Anthropic, Google, Mistral, OpenRouter, Reka, Groq, Together, Ollama, AI21, Cohere, Aleph-Alpha, HuggingfaceHub), with a built-in model performance benchmark.

Clarifai Evals is an SDK for evaluating AI models, providing a structured framework to benchmark model performance using predefined and custom evaluation templates.

Pytorch based library for robust prototyping, standardized benchmarking, and effortless experiment management

Benchmark dataset for learning dynamical systems from data

The official implementation of the WelQrate dataset and benchmark

Optimization benchmarks, both synthetic and practical.

VectorDBBench is not just an offering of benchmark results for mainstream vector databases and cloud services, it's your go-to tool for the ultimate performance and cost-effectiveness comparison. Designed with ease-of-use in mind, VectorDBBench is devised to help users, even non-professionals, reproduce results or test new systems, making the hunt for the optimal choice amongst a plethora of cloud services and open-source vector databases a breeze.

Create, Run and Benchmark DVC Pipelines in Python

AI Maintainer Agent Harness for our benchmarking and Marketplace API and platform

Quisby is a data processing and visualization tool for benchmark testing.

Preprocessing scripts for the DRAGON benchmark

LLM Benchmarking tool for OLLAMA

Benchmarks library, based on the package teneva, for testing multivariate approximation and optimization methods

RL4CO: an Extensive Reinforcement Learning for Combinatorial Optimization Benchmark

Unfied Semi-Supervised Learning Benchmark

A Python package for automated ML model benchmarking and comparison

Clinical Trial Omics Database for machine learning benchmarking

Quick and easy benchmarking for any command's CPU, memory, disk usage and runtime.

Video generation benchmark