🎩 You're Invited:Meet the Socket team at Black Hat in Las Vegas, August 3-6.RSVP →
Sign In

harness-evolver

Package Overview
Dependencies
Maintainers
1
Versions
104
Alerts
File Explorer

Advanced tools

License
Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

harness-evolver - npm Package Compare versions

Comparing version
6.3.0
 to
6.3.1
+1
-1
.claude-plugin/plugin.json
{
"name": "harness-evolver",
"description": "LangSmith-native autonomous agent optimization — evolves LLM agent code using multi-agent proposers, LangSmith experiments, and git worktrees",
"version": "6.3.0",
"version": "6.3.1",
"author": {

@@ -6,0 +6,0 @@ "name": "Raphael Valdetaro"

+1
-1
package.json
{
"name": "harness-evolver",
"version": "6.3.0",
"version": "6.3.1",
"description": "LangSmith-native autonomous agent optimization for Claude Code",

@@ -5,0 +5,0 @@ "author": "Raphael Valdetaro",

+16
-15
README.md

@@ -55,24 +55,25 @@ <p align="center">

Tested on a RAG agent (Agno framework, Gemini 3.1 Flash Lite, light mode):
```mermaid
xychart-beta
title "Best Score Over Evolution Iterations"
x-axis ["base", "v001", "v002", "v003", "v004", "v005", "v006", "v007", "v008", "v009"]
title "agno-deepknowledge: 0.575 → 1.000 (+74%)"
x-axis ["base", "v001", "v002", "v003", "v004", "v005", "v006", "v007"]
y-axis "Correctness" 0 --> 1
line [0.31, 0.48, 0.52, 0.52, 0.67, 0.71, 0.71, 0.71, 0.79, 0.84]
line [0.575, 0.575, 0.950, 0.950, 0.950, 0.950, 0.950, 1.0]
bar [0.575, 0.333, 0.950, 0.720, 0.875, 0.680, 0.880, 1.0]
```
| Iter | Score | Merged? | What happened |
| Iter | Score | Merged? | What the proposer did |
|---|---|---|---|
| baseline | 0.31 | — | Broken tool calls, hallucinations, no error handling |
| v001 | 0.48 | Yes | Fixed input parsing, added retry logic (+0.17) |
| v002 | 0.52 | Yes | Prompt rewrite to reduce hallucinations (+0.04) |
| v003 | 0.49 | **No** | Attempted retrieval change — regressed, rejected by gate |
| v004 | 0.67 | Yes | Architect triggered: chain → ReAct restructure (+0.15) |
| v005 | 0.71 | Yes | Output validation + citation grounding (+0.04) |
| v006 | 0.68 | **No** | Tried fewer tool calls — broke edge cases, rejected |
| v007 | 0.70 | **No** | Prompt tweak — within noise margin, not merged |
| v008 | 0.79 | Yes | Evolution memory insight: combined v003's retrieval with v005's validation (+0.08) |
| v009 | 0.84 | Yes | Fine-tuned rubric alignment from judge feedback (+0.05) |
| baseline | 0.575 | — | Original agent — hallucinations, broken tool calls, no retry logic |
| v001 | 0.333 | Yes | Anti-hallucination prompt (100% correct when API responded, but 60% hit rate limits) |
| v002 | 0.950 | Yes | **Breakthrough**: inlined 17-line KB into prompt, eliminated vector search entirely. 5.7x faster, zero rate limits |
| v003 | 0.720 | **No** | Attempted hybrid retrieval — regressed, rejected by constraint gate |
| v004 | 0.875 | **No** | Response completeness fix — improved one case but regressed others |
| v005 | 0.680 | **No** | Reduced tool calls — broke edge cases, rejected |
| v006 | 0.880 | Yes | Evolution memory insight: combined v001's anti-hallucination with one-shot example from archive |
| v007 | 1.000 | Yes | One-shot example injection + rubric-aligned responses — perfect on held-out |
Real pattern: initial jump → plateau → architectural breakthrough → small gains → stagnation → memory-driven recovery. Regressions rejected automatically. Not every iteration improves — that's the point of gate checks.
The line shows best score (only goes up — regressions aren't merged). The bars show each candidate's raw score. 4 merged, 3 rejected by gate checks. Not every iteration improves — that's the point.

@@ -79,0 +80,0 @@ ---

+5
-4
skills/evolve/SKILL.md

@@ -134,3 +134,4 @@ ---

```bash
SRC="$(pwd)"
SRC="$(git rev-parse --show-toplevel)"
[ -n "$PROJECT_DIR" ] && SRC="$SRC/$PROJECT_DIR"
# If langsmith-tracing companion is installed, proposer traces nest under iteration:

@@ -185,3 +186,3 @@ [ -n "$ITER_DOTTED_ORDER" ] && export CC_LANGSMITH_PARENT_DOTTED_ORDER="$ITER_DOTTED_ORDER"

[ -n "$PROJECT_DIR" ] && WT_PROJECT="$WT/$PROJECT_DIR"
$EVOLVER_PY $TOOLS/run_eval.py --config "$(pwd)/.evolver.json" --worktree-path "$WT_PROJECT" --experiment-prefix v{NNN}-{id} --concurrency $CONCURRENCY --timeout $TIMEOUT $SAMPLE &
$EVOLVER_PY $TOOLS/run_eval.py --config "$SRC/.evolver.json" --worktree-path "$WT_PROJECT" --experiment-prefix v{NNN}-{id} --concurrency $CONCURRENCY --timeout $TIMEOUT $SAMPLE &
done

@@ -191,3 +192,3 @@ wait # CRITICAL: wait for ALL evals before judge

Note: always pass `--config` with **absolute path** (`$(pwd)/.evolver.json`). The Bash tool's CWD may differ from the project root, causing relative paths to fail silently.
Note: `$SRC` is set via `git rev-parse --show-toplevel` at the start of each iteration — always resolves to the git root regardless of CWD drift.

@@ -310,3 +311,3 @@ **Auto-spawn LLM-as-judge** — check if LLM evaluators are configured and automatically spawn the evaluator agent. Do NOT leave this as a manual step for the user:

```bash
$EVOLVER_PY $TOOLS/cleanup_worktrees.py --dir "$(pwd)"
$EVOLVER_PY $TOOLS/cleanup_worktrees.py --dir "$SRC"
```

@@ -313,0 +314,0 @@