braintrust

Braintrust

A Python library for logging data to Braintrust. braintrust is distributed as a library on PyPI. It is open source and available on GitHub.

Quickstart

Install the library with pip.

pip install braintrust

Then, run a simple experiment with the following code (replace YOUR_API_KEY with your Braintrust API key):

from braintrust import Eval

def is_equal(expected, output):
    return expected == output

Eval(
  "Say Hi Bot",
  data=lambda: [
      {
          "input": "Foo",
          "expected": "Hi Foo",
      },
      {
          "input": "Bar",
          "expected": "Hello Bar",
      },
  ],  # Replace with your eval dataset
  task=lambda input: "Hi " + input,  # Replace with your LLM call
  scores=[is_equal],
)