Benchmarking

Guidance platform for deploying and managing large language models.

A web interaction benchmark for reinforcement learning.

🦜💪 Flex those feathers!

Benchmark resources usage

Pyinfer is a model agnostic Python utility tool for ML developers and researchers to benchmark model inference statistics.

Create charts from FIO storage benchmark tool output

Unfied Semi-Supervised Learning Benchmark

I/O profiler for deep learning python apps. Specifically for dlio_benchmark.

RAG Evaluation Benchmark and Evaluation Library

A library for training and benchmarking RL agents on graphs.

Benchmarking Likelihood Ratio systems

A benchmarking suite for robust audio watermarking.

SIBERIA (SIgned BEnchmarks foR tIme series Analysis) provides maximum-entropy null models and validation methods for signed networks derived from time series.

A simple loop execution time measurement tool.

🚀AI-Powered API Testing CLI - Generate tests from natural language, interactive builder, performance benchmarking, Postman import

Benchmarks library, based on the package teneva, for testing multivariate approximation and optimization methods

A GEMSEO-based package to benchmark optimization algorithms.

OpenMMLab Model Pretraining Toolbox and Benchmark

Benchmark orchestrator for the xemu original Microsoft Xbox emulator

Benchmarking library for generative algorithms

Open source remote sensing dataset with benchmarks

llama-index packs rag_evaluator integration

Optimum-Benchmark is a unified multi-backend utility for benchmarking Transformers, Timm, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.

Package for easy ligand preparation and docking with AutoDock Vina against a highly-curated selection of 58 protein targets. One of the three components of the DOCKSTRING bundle (package, dataset, and benchmark tasks).

Easily benchmark PyTorch model FLOPs, latency, throughput, max allocated memory and energy consumption in one go.

Python Interface to lfk-mp-benchmark

A Python interface for the USEPA Benchmark dose modeling software (BMDS)

Temporal Graph Benchmark project repo

A simple ANN benchmark tools

PyAnaDroid: A replicable, fully-customizable execution pipeline foranalyzing and benchmarking Android Applications

Benchmark functions for Bayesian optimization

Benchmark Python Scripts

A comprehensive, open-source LLM evaluation framework for testing and benchmarking AI models

A simple benchmarking library

Evolution Gym: A benchmark for developing and evaluating soft robot co-design algorithms.

Fibber is a benchmarking suite for adversarial attacks on text classification.

Packaged data modules for multiview learning benchmarks

Universal reusable Python utils for the modern human.

Pytorch based library for robust prototyping, standardized benchmarking, and effortless experiment management

Advanced benchmarking for machine learning models.

Easily benchmark Machine Learning models on selected tasks and datasets - with PyTorch

Collections of robotics environments geared towards benchmarking multi-task and meta reinforcement learning.

Python Optimization Benchmarking Functions

LLM Benchmarking tool for OLLAMA

a python package to benchmarks algorithms against various datasets

Use LLMs to get classification risk scores on tabular tasks.

Benchmark Tool for CaddoBenchmark Project

Python Cache Hierarchy Simulator

coNstructiOn pRoject Manager

Download NLP4BIA benchmarks and load datasets in their format