Test your LLM-powered apps with a TypeScript-native, Vitest-based eval runner. No API key required.
Evalite is still an experimental project. I'm actively working on it, and for now am pushing breaking changes.
If you run into any unexpected behavior:
node_modules/.evalite folder.evalite to the latest version.If, after that, you run into unexpected behavior, report an issue.
You can run Evalite in watch mode by running evalite watch:
evalite watch
This will watch for changes to your .eval.ts files and re-run the evals when they change.
[!IMPORTANT]
I strongly recommend implementing a caching layer in your LLM calls when using watch mode. This will keep your evals running fast and avoid burning through your API credits.
You can run specific files by passing them as arguments:
evalite my-eval.eval.ts
This also works for watch mode:
evalite watch my-eval.eval.ts
FAQs
Test your LLM-powered apps with a TypeScript-native, Vitest-based eval runner. No API key required.
The npm package evalite receives a total of 4,009 weekly downloads. As such, evalite popularity was classified as popular.
We found that evalite demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
As vulnerability data bottlenecks grow, the federal government is formally investigating NIST’s handling of the National Vulnerability Database.
Research
Security News
Socket’s Threat Research Team has uncovered 60 npm packages using post-install scripts to silently exfiltrate hostnames, IP addresses, DNS servers, and user directories to a Discord-controlled endpoint.
Security News
TypeScript Native Previews offers a 10x faster Go-based compiler, now available on npm for public testing with early editor and language support.