skill-maker

skill-maker

An Agent Skill that creates other agent skills. It guides an AI coding agent through the full lifecycle: intent capture, drafting a SKILL.md, running an eval loop with subagents, refining based on grading signals, and optimizing the description for trigger accuracy.

Visit skill-maker.pages.dev for an interactive overview of how it works, benchmark results, and quick-start install commands.

The eval loop is the core — it spawns isolated subagents per test case, grades assertions with bundled Bun TypeScript scripts, aggregates benchmarks, and iterates until pass_rate plateaus (delta < 2% for 3 consecutive iterations) or hits 20 iterations.

What's included

skill-maker/
├── SKILL.md                        # Main skill instructions
├── scripts/
│   ├── grade.ts                    # Grade assertions against eval outputs
│   ├── aggregate-benchmark.ts      # Aggregate grading into benchmark.json
│   ├── detect-plateau.ts           # Detect pass_rate plateau across iterations
│   └── validate-skill.ts           # Validate SKILL.md against the spec
├── references/
│   ├── schemas.md                  # JSON schemas for all eval artifact types
│   └── spec-summary.md             # Quick reference of the Agent Skills spec
├── assets/
│   └── skill-template.md           # Starter template with {{PLACEHOLDER}} markers
└── evals/
    └── evals.json                  # Test prompts with quality-focused assertions

Prerequisites

• Bun — required for bunx and all bundled scripts

Quick install

bunx skill-maker install

This installs the skill-maker skill to ~/.agents/skills/ and auto-detects any AI coding clients (Claude Code, OpenCode) to install there too.

Install example skills

skill-maker includes 9 example skills built with the eval loop. Install specific ones alongside skill-maker:

bunx skill-maker install pdf-toolkit code-reviewer

Or install everything:

bunx skill-maker install --all

Available skills

Skill	What it does
skill-maker	Creates other agent skills with eval-driven dev
api-doc-generator	Generates API documentation from code
changelog-generator	Creates changelogs from git history
code-reviewer	Reviews code for quality, bugs, and best practices
database-migration	Creates safe database migration scripts
error-handling	Adds comprehensive error handling patterns
git-conventional-commits	Writes conventional commit messages
monitoring-setup	Sets up application monitoring and alerting
pdf-toolkit	Extracts text, tables, and images from PDFs with OCR
pr-description	Writes detailed pull request descriptions

Run bunx skill-maker list to see all available skills.

Install options

# Force install to a specific client
bunx skill-maker install --client claude

# Also install to the current project (./agents/skills/)
bunx skill-maker install --local

# Combine flags
bunx skill-maker install pdf-toolkit --client opencode --local

Where skills are installed

Skills are always installed to ~/.agents/skills/. The CLI also auto-detects and installs to any client directories that exist on your system:

Client	Directory	Detected by
Generic	~/.agents/skills/	Always
Claude Code	~/.claude/skills/	~/.claude/ exists
OpenCode	~/.config/opencode/skills/	~/.config/opencode/ exists

Manual installation

If you prefer not to use bunx, clone and copy manually:

git clone https://github.com/accolver/skill-maker.git
cd skill-maker
mkdir -p ~/.agents/skills
cp -r skill-maker ~/.agents/skills/skill-maker

Verify installation

Run the bundled validator to confirm the skill is correctly structured:

bun run skill-maker/scripts/validate-skill.ts skill-maker

Expected output includes "valid": true with zero errors.

Usage

Once installed, ask your coding agent to create a skill. The skill triggers on prompts like:

• "Create a skill for writing git commit messages"
• "Build a SKILL.md that helps with data pipeline validation"
• "Make a reusable agent workflow for deploying to AWS"
• "Package this debugging process as a skill"

The agent will follow the 5-phase workflow automatically:

• Capture intent — asks clarifying questions about what the skill should do
• Draft — generates the SKILL.md, scripts, references, and assets
• Eval loop — runs test cases with and without the skill, grades outputs, detects plateau
• Refine — improves the skill based on which assertions failed
• Finalize — runs validation, optimizes the description, installs the skill

Benchmark results

Skills built with skill-maker were evaluated against unguided agents across 9 domains. Each skill went through the full eval loop: isolated subagent pairs (with-skill vs without-skill), assertion grading, and iteration until plateau.

Metric	Value
Skills evaluated	9
Total eval assertions	213
With-skill pass rate	100%
Average without-skill rate	23.9%
Average improvement	+76.1%
Average iterations to 100%	2.2

Per-skill results

Skill	With Skill	Without	Delta
database-migration	100%	4.2%	+95.8%
pdf-toolkit	100%	4.2%	+95.8%
error-handling	100%	8.3%	+91.7%
api-doc-generator	100%	16.7%	+83.3%
pr-description	100%	20.8%	+79.2%
changelog-generator	100%	20.8%	+79.2%
monitoring-setup	100%	26.1%	+73.9%
code-reviewer	100%	41.7%	+58.3%
git-conventional-commits	100%	72.3%	+27.7%

Skills add the most value where agents have knowledge but lack structure: output formatting, safety checklists, comprehensive coverage, and convention-specific rules consistently fail without skill guidance.

See examples/README.md for detailed per-skill breakdowns, convergence charts, and guidance on choosing high-delta skill use cases.

Self-evaluation

skill-maker was also tested on itself (meta-evaluation):

Metric	Score
with_skill pass rate	100%
without_skill rate	57.3%
Delta	+42.7%
Plateau reached at	Iter 6

See skill-maker-workspace/FINAL-BENCHMARK.md for the full iteration history.

License

MIT