🎩 You're Invited:Meet the Socket team at Black Hat in Las Vegas, August 3-6.RSVP →
five46 - npm Package Compare versions
+18
-14
| { | ||
| "name": "five46", | ||
| "version": "0.1.0", | ||
| "description": "BYOK, fully local agentic E2E and API testing: point five46 at a live URL or API and a goal, an LLM (your own OpenAI/Anthropic/Gemini/Groq/Bedrock key) drives a real local Playwright browser or real HTTP requests toward it, and writes a real, standalone, re-runnable Playwright/node:test spec on success. No cloud sandbox, no tunneling.", | ||
| "version": "0.1.1", | ||
| "description": "Autonomous AI testing agent that verifies your app or API actually works while you're still building it. Give it a plain-English goal, your own LLM key (OpenAI/Anthropic/Gemini/Groq/Bedrock) drives the real thing locally, and a real standalone Playwright/node:test spec is captured as a permanent regression test afterward. Fully local, BYOK, no cloud sandbox.", | ||
| "homepage": "https://github.com/sekharsdet/five46#readme", | ||
@@ -34,2 +34,12 @@ "repository": { | ||
| "keywords": [ | ||
| "ai-agent", | ||
| "autonomous-agent", | ||
| "ai-testing", | ||
| "autonomous-testing", | ||
| "agentic", | ||
| "agentic-testing", | ||
| "developer-tools", | ||
| "dev-testing", | ||
| "llm", | ||
| "byok", | ||
| "e2e", | ||
@@ -41,12 +51,9 @@ "e2e-testing", | ||
| "api-testing", | ||
| "qa", | ||
| "qa-automation", | ||
| "codegen", | ||
| "test-generation", | ||
| "self-healing-tests", | ||
| "playwright", | ||
| "playwright-tests", | ||
| "ai-agent", | ||
| "ai-testing", | ||
| "autonomous-testing", | ||
| "agentic", | ||
| "llm", | ||
| "byok", | ||
| "qa", | ||
| "qa-automation", | ||
| "cli", | ||
@@ -60,6 +67,3 @@ "mcp", | ||
| "groq", | ||
| "bedrock", | ||
| "codegen", | ||
| "test-generation", | ||
| "self-healing-tests" | ||
| "bedrock" | ||
| ], | ||
@@ -66,0 +70,0 @@ "license": "MIT", |
+73
-29
@@ -8,12 +8,16 @@ # five46 | ||
| **AI-powered, BYOK, fully local agentic E2E and API testing for Playwright — no cloud sandbox, no data leaving your machine except the LLM call itself.** | ||
| **An autonomous AI testing agent that verifies your app or API actually works while you're still building it — fully local, using your own LLM key.** | ||
| Point five46 at a real running page (or API) and a plain-English goal — "log in and confirm the dashboard loads," "create a user via POST, then confirm it via GET." An LLM — using your own OpenAI, Anthropic, Gemini, Groq, or AWS Bedrock key — drives a real local Playwright browser or real HTTP requests toward that goal, one action at a time, and writes a real, standalone, re-runnable Playwright (or `node:test`) spec on success. No five46, no LLM, and no network call back to us involved in re-running the generated test — it's just ordinary Playwright code you own outright. | ||
| You just changed something, and you want to know — right now, against the real running thing — whether it actually works, without first writing a test yourself. Give five46 a plain-English goal — "log in and confirm the dashboard loads," "create a user via POST, then confirm it via GET" — and an LLM, using your own OpenAI, Anthropic, Gemini, Groq, or AWS Bedrock key, drives your real app or real API, one real action at a time, and tells you honestly whether it worked, with a root-cause hypothesis if it didn't. Once it does, that exact run is captured as a real, standalone Playwright (or `node:test`) spec you keep — so the same check that helped you while you were building the feature becomes a permanent regression test afterward, with no five46 or LLM involved in ever running it again. | ||
| > **Status:** early proof of concept, verified end-to-end against real live LLM keys across dozens of real-world sites and APIs. | ||
| If five46 is useful to you, a ⭐ on [GitHub](https://github.com/sekharsdet/five46) helps other people find it — much appreciated! | ||
| ## Why five46, and how it's different | ||
| Most AI-driven test-generation tools run in a cloud sandbox: your app's traffic, screenshots, and DOM leave your machine and go through a third-party service you don't control. five46 is the opposite bet — **everything runs on your laptop**, using a key you already pay for, and the *only* thing that ever leaves your machine is the text sent to your chosen LLM provider on each step (always disclosed, never hidden). If your organization can't adopt a cloud-hosted AI testing platform for compliance or trust reasons, this is built for exactly that constraint. | ||
| Most testing tools assume you already have a suite to run. five46 is built for the moment *before* that — mid-feature, before a test exists at all. Point it at what you're building, describe the outcome you expect in plain English, and keep re-running it as you keep changing code; once it's solid, the run it just did becomes your regression test, not a separate thing you write afterward. | ||
| Most AI-driven test-generation tools also run in a cloud sandbox: your app's traffic, screenshots, and DOM leave your machine and go through a third-party service you don't control. five46 is the opposite bet — **everything runs on your laptop**, using a key you already pay for, and the *only* thing that ever leaves your machine is the text sent to your chosen LLM provider on each step (always disclosed, never hidden). If your organization can't adopt a cloud-hosted AI testing platform for compliance or trust reasons, this is built for exactly that constraint. | ||
| It's also not a black box: every run ends with a real `.spec.ts`/`.test.mjs` file you can read, diff, commit to your repo, and run in CI with plain `npx playwright test` — no vendor lock-in, no proprietary runner. | ||
@@ -27,7 +31,7 @@ | ||
| Nothing but the LLM calls ever leaves your machine. | ||
| - **Browser and API testing** — drive a real Chromium browser, or drive | ||
| real HTTP requests directly, from the same agentic engine. | ||
| - **Real, standalone output** — every successful run writes a plain | ||
| Playwright `.spec.ts` (or `node:test` script for API tests) you can | ||
| re-run any time, with no five46 or LLM involved. | ||
| - **Browser and API testing** — drive a real Chromium browser, or drive | ||
| real HTTP requests directly, from the same agentic engine. | ||
| - **Session reuse** — log in once, capture the session, reuse it across | ||
@@ -70,2 +74,18 @@ runs without paying the LLM cost of logging in every time. | ||
| ```bash | ||
| npm install -g five46 | ||
| npm install --save-dev playwright @playwright/test # one-time, if your project doesn't already have it | ||
| npx playwright install chromium # one-time, downloads the browser | ||
| ``` | ||
| Or run it without installing globally: | ||
| ```bash | ||
| npx five46 test http://localhost:3000 --goal "log in and confirm the dashboard loads" | ||
| ``` | ||
| <details> | ||
| <summary>Building from source instead (for contributing to five46 itself)</summary> | ||
| ```bash | ||
| git clone https://github.com/sekharsdet/five46.git | ||
@@ -75,7 +95,7 @@ cd five46 | ||
| npm run build | ||
| npm install --save-dev playwright @playwright/test # one-time | ||
| npx playwright install chromium # one-time, downloads the browser | ||
| node dist/cli.js test http://localhost:3000 --goal "..." | ||
| ``` | ||
| </details> | ||
| ## Configuration | ||
@@ -86,3 +106,3 @@ | ||
| ```bash | ||
| node dist/cli.js config | ||
| five46 config | ||
| ``` | ||
@@ -100,6 +120,37 @@ | ||
| ### Getting a key | ||
| Don't have a key yet? Pick whichever's easiest to get, or whichever you | ||
| already use — five46 calls one small, cheap model per provider on every | ||
| step (never a "flagship" model), so per-run cost is low regardless of | ||
| which one you pick: | ||
| | Provider | Get a key at | Notes | | ||
| |---|---|---| | ||
| | **Gemini** | [aistudio.google.com](https://aistudio.google.com/apikey) → "Get API key" | Free tier, no credit card required — the fastest path to a first successful run. | | ||
| | **Groq** | [console.groq.com/keys](https://console.groq.com/keys) → "Create API Key" | Free tier, no credit card required, generous rate limits. | | ||
| | **OpenAI** | [platform.openai.com/api-keys](https://platform.openai.com/api-keys) → "Create new secret key" | Account creation is free, but a key can't make real calls until you add a payment method — no meaningful free tier. | | ||
| | **Anthropic** | [console.anthropic.com/settings/keys](https://console.anthropic.com/settings/keys) → "Create Key" | Same shape as OpenAI — you can browse the console for free, but need billing set up before a key actually works. | | ||
| | **AWS Bedrock** | No key — see below | Uses your existing AWS credentials instead of an API key. | | ||
| The model each provider calls: `gpt-4o-mini` (OpenAI), `claude-3-5-haiku-latest` | ||
| (Anthropic), `gemini-flash-latest` (Gemini), `llama-3.3-70b-versatile` (Groq), | ||
| `anthropic.claude-3-5-haiku-20241022-v1:0` (Bedrock). | ||
| **AWS Bedrock is different — there's no key to paste in:** | ||
| 1. In the [Bedrock console](https://console.aws.amazon.com/bedrock/home) → | ||
| **Model access**, request/enable access to the Claude model above, in | ||
| the region you plan to use. | ||
| 2. Make sure AWS credentials are available the normal way — five46 relies | ||
| on the standard AWS SDK credential chain, same as the AWS CLI: | ||
| `aws configure`, `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` env vars, or | ||
| an IAM role. | ||
| 3. Run `five46 config`, choose `bedrock`, and enter your **region** | ||
| (e.g. `us-east-1`) when prompted — not a key. | ||
| ## Quick start | ||
| ```bash | ||
| node dist/cli.js test http://localhost:3000 --goal "log in and confirm the dashboard loads" | ||
| five46 test http://localhost:3000 --goal "log in and confirm the dashboard loads" | ||
| ``` | ||
@@ -143,4 +194,4 @@ | ||
| node dist/cli.js login https://your-app.example.com/login --goal "log in" --out session.json | ||
| node dist/cli.js test https://your-app.example.com/dashboard --goal "..." --storage-state session.json | ||
| five46 login https://your-app.example.com/login --goal "log in" --out session.json | ||
| five46 test https://your-app.example.com/dashboard --goal "..." --storage-state session.json | ||
| ``` | ||
@@ -162,3 +213,3 @@ | ||
| ```bash | ||
| node dist/cli.js api https://api.your-app.example.com --goal "create a user, then fetch it back and confirm the name matches" | ||
| five46 api https://api.your-app.example.com --goal "create a user, then fetch it back and confirm the name matches" | ||
| ``` | ||
@@ -174,5 +225,5 @@ | ||
| ```bash | ||
| node dist/cli.js list # current directory | ||
| node dist/cli.js list ./tests # or any other directory | ||
| node dist/cli.js list --project checkout # only runs tagged with this project | ||
| five46 list # current directory | ||
| five46 list ./tests # or any other directory | ||
| five46 list --project checkout # only runs tagged with this project | ||
| ``` | ||
@@ -188,3 +239,3 @@ | ||
| ```bash | ||
| node dist/cli.js diff five46-agent-abc123.spec.ts five46-agent-def456.spec.ts | ||
| five46 diff five46-agent-abc123.spec.ts five46-agent-def456.spec.ts | ||
| ``` | ||
@@ -199,3 +250,3 @@ | ||
| ```bash | ||
| node dist/cli.js test http://localhost:3000 --goal "..." --repeat 5 | ||
| five46 test http://localhost:3000 --goal "..." --repeat 5 | ||
| ``` | ||
@@ -221,3 +272,3 @@ | ||
| ```bash | ||
| node dist/cli.js test --goal "..." --project checkout | ||
| five46 test --goal "..." --project checkout | ||
| ``` | ||
@@ -232,3 +283,3 @@ | ||
| ```bash | ||
| node dist/cli.js test http://localhost:3000 --goal "..." --record-video | ||
| five46 test http://localhost:3000 --goal "..." --record-video | ||
| ``` | ||
@@ -243,3 +294,3 @@ | ||
| ```bash | ||
| node dist/cli.js test http://localhost:3000 --goal "..." --structured-plan | ||
| five46 test http://localhost:3000 --goal "..." --structured-plan | ||
| ``` | ||
@@ -259,3 +310,3 @@ | ||
| npm install --save-dev @modelcontextprotocol/sdk zod # one-time | ||
| node dist/cli.js mcp | ||
| five46 mcp | ||
| ``` | ||
@@ -270,9 +321,2 @@ | ||
| ## Roadmap | ||
| Not yet built: a multi-file/dependency-graph backend-test model, auto- | ||
| refresh login (session expiry mid-run), MCP exposure for `five46 login`, | ||
| concurrent `--repeat` execution, and a cheaper/faster model for per-step | ||
| decisions with a stronger model reserved for planning. | ||
| ## Development | ||
@@ -279,0 +323,0 @@ |
No alert changes
Improved metrics
- Total package byte prevSize
458182
0.62%- Number of lines in readme file
317
16.12%No dependency changes