Benchmarking

Python benchmark suite

Electrical Power System Benchmark Models.

The official SWE-smith package - A toolkit for generating software engineering training data at scale.

Experiment management and benchmark tools for mathematical optimization

AI Benchmark is an open source python library for evaluating AI performance of various hardware platforms, including CPUs, GPUs and TPUs.

A comprehensive benchmark and code base for Image manipulation and localization.

A simple and easy-to-use Python benchmarking library

A lightweight toolkit for evaluating LLMs based on OpenCompass.

Benchmark for language models

Minimal Python library to connect to LLMs (OpenAI, Anthropic, Google, Mistral, OpenRouter, Reka, Groq, Together, Ollama, AI21, Cohere, Aleph-Alpha, HuggingfaceHub), with a built-in model performance benchmark.

A pioneering unified platform designed to systematize and accelerate deep learning research in spectroscopy.

pygmtools provides graph matching solvers in Python API and supports numpy and pytorch backends. pygmtools also provides dataset API for standard graph matching benchmarks.

Benchmark tools for robotframework

Count floating-point operations in Python code & benchmark relative flop costs.

A package for submitting benchmarking scripts on OSCAR.

The Feel++ Benchmarking Project

Reproducible Benchmark for Everyone

Benchmark performance of **any Foundation Model (FM)** deployed on **any AWS Generative AI service**, be it **Amazon SageMaker**, **Amazon Bedrock**, **Amazon EKS**, or **Amazon EC2**. The FMs could be deployed on these platforms either directly through `FMbench`, or, if they are already deployed then also they could be benchmarked through the **Bring your own endpoint** mode supported by `FMBench`.

Indexer for GZIP specially built for DLIO Profiler.

LLM Benchmark

Continuous Benchmarking (CB) Framework

Platform Performance Benchmarking

A study to benchmark whisper based ASRs in Malayalam

Benchmark sagemaker serverless endpoints for cost and performance

OGBench: Benchmarking Offline Goal-Conditioned RL

A tool for Behavior benchmARKing

Benchmarking the performance of agents far and wide, regardless of how they are set up and how they work

Small Benchmarks for LM Agents

Measure the energy used by your MPI+Python applications.

Python Automatic Forecasting

Rotation Detection Toolbox and Benchmark

A Python library for managing, processing, and benchmarking datasets in SQLite databases for AI pipelines and LLM prompt engineering.

A powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serving systems.

AssayInspector: A Python package for diagnostic assessment of data consistency in molecular datasets.

A python package for accurate benchmarking and speed comparisons

The specs summarize utilities for computer instance

AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents

VectorDBBench is not just an offering of benchmark results for mainstream vector databases and cloud services, it's your go-to tool for the ultimate performance and cost-effectiveness comparison. Designed with ease-of-use in mind, VectorDBBench is devised to help users, even non-professionals, reproduce results or test new systems, making the hunt for the optimal choice amongst a plethora of cloud services and open-source vector databases a breeze.

RL4CO: an Extensive Reinforcement Learning for Combinatorial Optimization Benchmark

A Comprehensive Benchmark of Deep Model Fusion

Fix Inventory Compliance Benchmarks and Checks

rliable: Reliable evaluation on reinforcement learning and machine learning benchmarks.

Benchmark functions that returns total space, mem, cpu given input size and parameters for the CWL workflows

Repostory of Protein Benchmarking and Modeling

Comprehensive benchmarking of protein-ligand structure prediction methods

OmniGenBench: A comprehensive toolkit for genome analysis benchmarking.

Loop Kernel Analysis and Performance Modeling Toolkit

Set of robot URDFs for benchmarking and developed examples.

Isaac Sim components for benchmarking

Adapters for Running and Tracking Benchmarks