homunculus-code
Advanced tools
+29
| # Homunculus — Development Guide | ||
| ## Release Checklist | ||
| Every change to this repo must complete these steps in order: | ||
| 1. **Code changes** — implement the feature/fix | ||
| 2. **README.md** — update What's New section + any affected docs | ||
| 3. **package.json** — bump version (semver: patch/minor/major) | ||
| 4. **git commit + push** — descriptive commit message | ||
| 5. **npm publish** — `npm publish` from repo root | ||
| 6. **GitHub release** — `gh release create vX.Y.Z` with release notes | ||
| 7. **assistant plan.md** — sync `~/assistant/projects/homunculus/plan.md` if applicable | ||
| ## Project Structure | ||
| - `bin/` — CLI entry points (init, upgrade, night) | ||
| - `core/` — Evolution scripts copied to user projects during init | ||
| - `commands/` — Slash commands copied to `.claude/commands/` during init | ||
| - `templates/` — CLAUDE.md, architecture.yaml, rules templates for init | ||
| - `docs/` — Long-form documentation | ||
| - `examples/reference/` — Real-world reference implementation from the author's system | ||
| ## Key Design Decisions | ||
| - **Zero dependencies** — Only Node.js stdlib + Claude CLI | ||
| - **File copy, not symlink** — init copies files so users can customize freely | ||
| - **Manifest-based upgrade** — `.manifest.json` tracks SHA256 hashes for diff-aware upgrades | ||
| - **Never touch user content** — upgrade never modifies CLAUDE.md, architecture.yaml, instincts, skills |
+11
-9
@@ -108,16 +108,18 @@ # Architecture: Goal-Tree-Driven Evolution | ||
| ``` | ||
| Tool usage → observe.sh (hook) → observations.jsonl | ||
| | | ||
| evaluate-session.js (SessionEnd) | ||
| | | ||
| v | ||
| instincts/personal/*.md | ||
| (confidence-scored patterns) | ||
| (tagged: suggested_mechanism + goal_path) | ||
| Tool usage → observe.sh (hook, noise-filtered) → observations.jsonl | ||
| | | | ||
| |→ reference-tracking.jsonl evaluate-session.js | ||
| (which instincts/skills | | ||
| are actually read) v | ||
| ┌── instincts/personal/*.md | ||
| │ (confidence-scored, mechanism-tagged) | ||
| ├── reports/memory-suggestions.jsonl | ||
| └── reports/research-queue.jsonl | ||
| ``` | ||
| Instincts are small behavioral patterns with: | ||
| - **Confidence scores** — increase with reinforcement, decay over time | ||
| - **Confidence scores** — increase with reinforcement, decay over time (14-day grace period) | ||
| - **Mechanism tag** — which implementation type fits best (hook/rule/skill/script/...) | ||
| - **Goal path** — which goal in the tree this serves | ||
| - **Semantic dedup** — new instincts can `supersede` older ones, auto-archiving them | ||
| - **Automatic archival** — archived once implemented by any mechanism | ||
@@ -124,0 +126,0 @@ |
+296
-85
| #!/usr/bin/env node | ||
| // evaluate-session.js — Extract instincts from session observations | ||
| // evaluate-session.js — Extract instincts, memory suggestions, and research topics from sessions | ||
| // Part of the Homunculus evolution pipeline | ||
| // | ||
| // v0.9.0: Three-layer extraction, dynamic daily cap, semantic dedup, Write Gate | ||
| // | ||
| // Flow: | ||
| // 1. Read observations.jsonl, analyze tool usage patterns | ||
| // 2. Determine if session is worth extracting (enough activity) | ||
| // 3. Use Claude CLI to extract behavioral patterns as instincts | ||
| // 1. Read observations.jsonl OR session transcript, analyze tool usage patterns | ||
| // 2. Check daily cooldown + dynamic limit based on session size | ||
| // 3. Use Claude CLI to extract behavioral patterns as instincts + memory + research | ||
| // 4. Write instincts to homunculus/instincts/personal/ | ||
| // 5. Write memory/research suggestions to homunculus/reports/ | ||
@@ -19,9 +22,16 @@ const fs = require('fs'); | ||
| const INSTINCTS_DIR = path.join(HOMUNCULUS_DIR, 'instincts', 'personal'); | ||
| const ARCHIVE_DIR = path.join(HOMUNCULUS_DIR, 'instincts', 'archived'); | ||
| const OBS_FILE = path.join(HOMUNCULUS_DIR, 'observations.jsonl'); | ||
| const REPORTS_DIR = path.join(HOMUNCULUS_DIR, 'reports'); | ||
| const COOLDOWN_FILE = path.join(HOMUNCULUS_DIR, '.cooldown.json'); | ||
| const SCAN_STATE_FILE = path.join(HOMUNCULUS_DIR, '.scan-state.json'); | ||
| // Extraction thresholds (auto-tunable) | ||
| // Extraction thresholds | ||
| const CONFIG = { | ||
| min_messages: parseInt(process.env.HOMUNCULUS_MIN_MESSAGES || '10'), | ||
| min_tool_repeats: parseInt(process.env.HOMUNCULUS_MIN_TOOL_REPEATS || '5'), | ||
| daily_limit: parseInt(process.env.HOMUNCULUS_DAILY_LIMIT || '3') | ||
| // Dynamic daily cap: scales with session size | ||
| daily_limit_thresholds: [0, 30, 60, 100], | ||
| daily_limit_values: [1, 2, 3, 5], | ||
| hard_cap: 5 | ||
| }; | ||
@@ -41,17 +51,87 @@ | ||
| // Count today's extractions to respect daily limit | ||
| function getTodayExtractions() { | ||
| const today = getDateString(); | ||
| // Dynamic daily limit based on session size | ||
| function getDailyLimit(msgCount) { | ||
| const { daily_limit_thresholds: thresholds, daily_limit_values: limits } = CONFIG; | ||
| for (let i = thresholds.length - 1; i >= 0; i--) { | ||
| if (msgCount >= thresholds[i]) return limits[i] || 1; | ||
| } | ||
| return 1; | ||
| } | ||
| // Cooldown state file (O(1) lookup instead of scanning all instinct files) | ||
| function getCooldown() { | ||
| try { | ||
| const files = fs.readdirSync(INSTINCTS_DIR); | ||
| return files.filter(f => { | ||
| const content = fs.readFileSync(path.join(INSTINCTS_DIR, f), 'utf8'); | ||
| return content.includes(today); | ||
| }).length; | ||
| return JSON.parse(fs.readFileSync(COOLDOWN_FILE, 'utf8')); | ||
| } catch { | ||
| return 0; | ||
| return { date: null, count: 0 }; | ||
| } | ||
| } | ||
| // Analyze observations to find patterns | ||
| function updateCooldown(count) { | ||
| const today = getDateString(); | ||
| fs.writeFileSync(COOLDOWN_FILE, JSON.stringify({ date: today, count })); | ||
| } | ||
| // Get existing instinct names (for duplicate check) | ||
| function getExistingInstincts() { | ||
| ensureDir(INSTINCTS_DIR); | ||
| return fs.readdirSync(INSTINCTS_DIR) | ||
| .filter(f => f.endsWith('.md')) | ||
| .map(f => f.replace(/\.md$/, '')); | ||
| } | ||
| // Get existing instincts with titles for semantic supersede detection | ||
| function getExistingInstinctsWithTitles() { | ||
| ensureDir(INSTINCTS_DIR); | ||
| const results = []; | ||
| for (const f of fs.readdirSync(INSTINCTS_DIR).filter(f => f.endsWith('.md'))) { | ||
| const name = f.replace(/\.md$/, ''); | ||
| try { | ||
| const content = fs.readFileSync(path.join(INSTINCTS_DIR, f), 'utf8'); | ||
| const titleMatch = content.match(/^# Instinct: (.+)/m); | ||
| const title = titleMatch ? titleMatch[1].trim() : name; | ||
| results.push(`${name}: ${title}`); | ||
| } catch { results.push(name); } | ||
| } | ||
| return results; | ||
| } | ||
| // Archive an instinct that has been superseded by a newer one | ||
| function archiveSupersededInstinct(name, supersededBy) { | ||
| ensureDir(ARCHIVE_DIR); | ||
| const src = path.join(INSTINCTS_DIR, `${name}.md`); | ||
| const dst = path.join(ARCHIVE_DIR, `${name}.md`); | ||
| if (!fs.existsSync(src)) return false; | ||
| let content = fs.readFileSync(src, 'utf8'); | ||
| content += `\n\n---\n**Archived:** ${new Date().toISOString()}\n**Covered-by:** ${supersededBy} (supersede)\n`; | ||
| fs.writeFileSync(dst, content); | ||
| fs.unlinkSync(src); | ||
| return true; | ||
| } | ||
| // Save memory suggestion to reports (non-invasive — user decides what to adopt) | ||
| function saveMemorySuggestion(entry) { | ||
| ensureDir(REPORTS_DIR); | ||
| const file = path.join(REPORTS_DIR, 'memory-suggestions.jsonl'); | ||
| const record = { | ||
| ...entry, | ||
| timestamp: new Date().toISOString(), | ||
| status: 'pending' | ||
| }; | ||
| fs.appendFileSync(file, JSON.stringify(record) + '\n'); | ||
| } | ||
| // Save research topic to reports | ||
| function saveResearchTopic(entry) { | ||
| ensureDir(REPORTS_DIR); | ||
| const file = path.join(REPORTS_DIR, 'research-queue.jsonl'); | ||
| const record = { | ||
| ...entry, | ||
| timestamp: new Date().toISOString(), | ||
| status: 'pending' | ||
| }; | ||
| fs.appendFileSync(file, JSON.stringify(record) + '\n'); | ||
| } | ||
| // Analyze observations.jsonl for tool patterns | ||
| function analyzeObservations() { | ||
@@ -69,3 +149,2 @@ if (!fs.existsSync(OBS_FILE)) { | ||
| // Count tool usage | ||
| const toolCounts = {}; | ||
@@ -80,3 +159,2 @@ for (const line of lines) { | ||
| // Find high-frequency tools | ||
| const frequentTools = Object.entries(toolCounts) | ||
@@ -94,85 +172,101 @@ .filter(([_, count]) => count >= CONFIG.min_tool_repeats) | ||
| frequent_tools: frequentTools, | ||
| tool_counts: toolCounts | ||
| tool_counts: toolCounts, | ||
| user_msg_count: lines.length // approximate | ||
| }; | ||
| } | ||
| // Extract instinct using Claude CLI | ||
| function extractInstinct(analysis) { | ||
| const toolSummary = analysis.frequent_tools | ||
| .map(([tool, count]) => `${tool}: ${count} times`) | ||
| .join(', '); | ||
| // Read architecture.yaml goal names for goal_path tagging | ||
| let goalHint = ''; | ||
| // Read goal tree names for goal_path tagging | ||
| function getGoalHint() { | ||
| try { | ||
| const archFile = path.join(BASE_DIR, 'architecture.yaml'); | ||
| if (fs.existsSync(archFile)) { | ||
| const archContent = fs.readFileSync(archFile, 'utf8'); | ||
| const goals = [...archContent.matchAll(/^ (\w+):\s*$/gm)].map(m => m[1]).filter(g => !['goals', 'metrics', 'agents', 'health_check', 'test_config'].includes(g)); | ||
| goalHint = goals.length > 0 ? `\nGoal tree top-level goals: ${goals.join(', ')}` : ''; | ||
| const content = fs.readFileSync(archFile, 'utf8'); | ||
| const goals = [...content.matchAll(/^ (\w+):\s*$/gm)] | ||
| .map(m => m[1]) | ||
| .filter(g => !['goals', 'metrics', 'agents', 'health_check', 'test_config'].includes(g)); | ||
| return goals.length > 0 ? goals.join(', ') : ''; | ||
| } | ||
| } catch {} | ||
| return ''; | ||
| } | ||
| const prompt = `Based on this session's tool usage patterns, extract ONE behavioral instinct that would be useful to remember for future sessions. | ||
| // Extract instincts + memory + research using Claude CLI | ||
| function extractFromSession(analysis) { | ||
| const toolSummary = analysis.frequent_tools | ||
| .map(([tool, count]) => `${tool}: ${count} times`) | ||
| .join(', '); | ||
| Tool usage: ${toolSummary} | ||
| Total observations: ${analysis.total_observations} | ||
| ${goalHint} | ||
| const goalHint = getGoalHint(); | ||
| const existingWithTitles = getExistingInstinctsWithTitles(); | ||
| Implementation mechanisms (pick the best fit for suggested_mechanism): | ||
| - hook: deterministic, every time, zero AI judgment (e.g. lint after edit) | ||
| - rule: path-scoped guidance for specific directories | ||
| - skill: reusable knowledge collection with eval spec | ||
| - script: periodic automation, no AI needed at runtime | ||
| - agent: isolated context, specialist role | ||
| - system: infrastructure-level | ||
| const prompt = `Analyze this Claude Code session and extract reusable behavioral patterns. | ||
| Write the instinct as a markdown file with this format: | ||
| ## Session Stats | ||
| - Observations: ${analysis.total_observations} | ||
| - Frequent tools: ${toolSummary} | ||
| ${goalHint ? `\n## Goal Tree (for goal_path)\n${goalHint}` : ''} | ||
| --- | ||
| name: descriptive-kebab-case-name | ||
| category: one of [coding, debugging, workflow, communication, tool-usage] | ||
| confidence: 0.7 | ||
| extracted: "${getDateString()}" | ||
| source: "session observation" | ||
| suggested_mechanism: hook|rule|skill|script|agent|system | ||
| goal_path: "top_level_goal.sub_goal or unrooted if no match" | ||
| --- | ||
| ## Existing Instincts (avoid duplicates) | ||
| ${existingWithTitles.length > 0 ? existingWithTitles.join('\n') : '(none yet)'} | ||
| ## Pattern | ||
| [Describe the behavioral pattern in 1-2 sentences] | ||
| ## Write Gate (must pass at least one to record) | ||
| An observation must meet at least one criterion to become an instinct: | ||
| - **Changes future behavior**: Would this pattern make future sessions different? | ||
| - **Commitment**: User explicitly stated a preference ("always use CLI not MCP") | ||
| - **Decision rationale**: Why A not B (key reasoning worth preserving) | ||
| - **Stable fact**: System knowledge unlikely to change | ||
| - **Explicit instruction**: User said "remember this" or "always do this" | ||
| ## When to Apply | ||
| [When should this pattern be used] | ||
| Observations that fail all criteria → no instinct (output empty line). | ||
| ## Anti-Patterns | ||
| [What to avoid] | ||
| ## Implementation Mechanisms (for suggested_mechanism) | ||
| hook (deterministic, every time) | rule (path-scoped guidance) | skill (reusable knowledge with eval) | script (periodic automation) | agent (isolated context specialist) | system (infrastructure) | ||
| IMPORTANT: Only output the markdown content, nothing else.`; | ||
| ## Output Format | ||
| try { | ||
| const result = execSync( | ||
| `claude -p "${prompt.replace(/"/g, '\\"')}" --model claude-sonnet-4-6 --max-budget-usd 0.50`, | ||
| { encoding: 'utf8', timeout: 30000 } | ||
| ); | ||
| ### A. Instincts (behavioral patterns) | ||
| Find 1-2 reusable patterns (confidence > 0.7, must pass Write Gate): | ||
| {"type": "instinct", "name": "kebab-case-name", "confidence": 0.8, "title": "Pattern title", "content": "Full instinct description", "suggested_mechanism": "hook|rule|skill|script|agent|system", "goal_path": "top_goal.sub_goal or unrooted", "supersedes": []} | ||
| // Extract name from frontmatter | ||
| const nameMatch = result.match(/name:\s*(.+)/); | ||
| if (!nameMatch) return null; | ||
| - supersedes: if this new pattern semantically replaces an existing instinct, list its name. Leave [] if unsure. | ||
| const name = nameMatch[1].trim().replace(/[^a-z0-9-]/g, '-'); | ||
| const filename = `${name}.md`; | ||
| const filepath = path.join(INSTINCTS_DIR, filename); | ||
| ### B. Memory Suggestions (important context to preserve) | ||
| Check if the session contains information worth long-term storage (not instincts, but facts): | ||
| - User preferences or corrections | ||
| - Project decisions or direction changes | ||
| - External resources discovered | ||
| - User background updates | ||
| if (fs.existsSync(filepath)) { | ||
| log(`Instinct ${filename} already exists, skipping`); | ||
| return null; | ||
| } | ||
| {"type": "memory", "category": "user|feedback|project|reference", "title": "Short title", "content": "Content to save"} | ||
| ensureDir(INSTINCTS_DIR); | ||
| fs.writeFileSync(filepath, result.trim() + '\n'); | ||
| log(`Extracted instinct: ${filename}`); | ||
| return filename; | ||
| ### C. Research Topics (mentioned but not explored) | ||
| Check for topics the user mentioned but didn't dive into: | ||
| - Questions deferred ("look into this later") | ||
| - Tools/tech mentioned but not explored | ||
| {"type": "research", "topic": "Topic name", "context": "Why this is interesting", "goal_path": "goal path or unrooted"} | ||
| ### Rules | ||
| - One JSON per line, must include "type" field | ||
| - No instinct if Write Gate fails, no memory if nothing worth saving, no research if nothing deferred | ||
| - Output ONLY JSON lines or empty lines, no other text`; | ||
| try { | ||
| const env = { ...process.env }; | ||
| delete env.CLAUDECODE; // Prevent recursive session | ||
| const model = process.env.HOMUNCULUS_HARVEST_MODEL || 'claude-sonnet-4-6'; | ||
| const result = execSync( | ||
| `claude --print --model ${model} --max-turns 1 --no-session-persistence`, | ||
| { | ||
| input: prompt, | ||
| encoding: 'utf8', | ||
| timeout: 120000, | ||
| env, | ||
| stdio: ['pipe', 'pipe', 'pipe'] | ||
| } | ||
| ).trim(); | ||
| return result; | ||
| } catch (err) { | ||
| log(`Extraction failed: ${err.message}`); | ||
| log(`Extraction failed: ${err.message?.slice(0, 100)}`); | ||
| return null; | ||
@@ -182,8 +276,98 @@ } | ||
| // Process extraction result — parse JSON lines and route to appropriate handlers | ||
| function processResult(result, { existing, todayCount, dynamicLimit }) { | ||
| if (!result) return { instincts: 0, memory: 0, research: 0 }; | ||
| const stats = { instincts: 0, memory: 0, research: 0 }; | ||
| for (const line of result.split('\n')) { | ||
| const trimmed = line.trim(); | ||
| if (!trimmed || !trimmed.startsWith('{')) continue; | ||
| let entry; | ||
| try { entry = JSON.parse(trimmed); } catch { continue; } | ||
| switch (entry.type) { | ||
| case 'instinct': { | ||
| if (todayCount + stats.instincts >= dynamicLimit) break; | ||
| if (!entry.name || !entry.content || (entry.confidence || 0) < 0.7) break; | ||
| const name = entry.name.replace(/[^a-z0-9-]/g, '-'); | ||
| if (existing.includes(name)) { | ||
| log(` Skip duplicate: ${name}`); | ||
| break; | ||
| } | ||
| // Handle supersedes | ||
| const supersedes = Array.isArray(entry.supersedes) ? entry.supersedes : []; | ||
| for (const oldName of supersedes) { | ||
| if (archiveSupersededInstinct(oldName, name)) { | ||
| log(` Superseded: ${oldName} → archived (replaced by ${name})`); | ||
| } | ||
| } | ||
| // Write instinct file | ||
| const mechanism = entry.suggested_mechanism || 'skill'; | ||
| const goalPath = entry.goal_path || 'unrooted'; | ||
| const supersedesNote = supersedes.length > 0 ? `\n**Supersedes:** ${supersedes.join(', ')}` : ''; | ||
| const content = `# Instinct: ${entry.title || name} | ||
| **Confidence:** ${entry.confidence} | ||
| **Source:** auto-learn (evaluate-session) | ||
| **Extracted:** ${new Date().toISOString()} | ||
| **Suggested mechanism:** ${mechanism} | ||
| **Goal path:** ${goalPath}${supersedesNote} | ||
| ${entry.content} | ||
| `; | ||
| ensureDir(INSTINCTS_DIR); | ||
| fs.writeFileSync(path.join(INSTINCTS_DIR, `${name}.md`), content); | ||
| stats.instincts++; | ||
| log(` Extracted instinct: ${name} (confidence: ${entry.confidence})`); | ||
| break; | ||
| } | ||
| case 'memory': { | ||
| if (entry.title && entry.content && entry.category) { | ||
| saveMemorySuggestion(entry); | ||
| stats.memory++; | ||
| log(` Memory suggestion: [${entry.category}] ${entry.title}`); | ||
| } | ||
| break; | ||
| } | ||
| case 'research': { | ||
| if (entry.topic) { | ||
| saveResearchTopic(entry); | ||
| stats.research++; | ||
| log(` Research topic: ${entry.topic}`); | ||
| } | ||
| break; | ||
| } | ||
| } | ||
| } | ||
| return stats; | ||
| } | ||
| // Main | ||
| function main() { | ||
| // Check daily limit | ||
| const todayCount = getTodayExtractions(); | ||
| if (todayCount >= CONFIG.daily_limit) { | ||
| log(`Daily limit reached (${todayCount}/${CONFIG.daily_limit}), skipping`); | ||
| // Idempotency: check scan state to avoid reprocessing | ||
| try { | ||
| if (fs.existsSync(SCAN_STATE_FILE)) { | ||
| const state = JSON.parse(fs.readFileSync(SCAN_STATE_FILE, 'utf8')); | ||
| const obsModified = fs.existsSync(OBS_FILE) ? fs.statSync(OBS_FILE).mtimeMs : 0; | ||
| if (state.last_obs_mtime && obsModified <= state.last_obs_mtime) { | ||
| log('Observations unchanged since last evaluation, skipping'); | ||
| return; | ||
| } | ||
| } | ||
| } catch {} | ||
| // Daily cooldown check | ||
| const cooldown = getCooldown(); | ||
| const today = getDateString(); | ||
| const todayCount = cooldown.date === today ? cooldown.count : 0; | ||
| if (todayCount >= CONFIG.hard_cap) { | ||
| log(`Hard cap reached (${todayCount}/${CONFIG.hard_cap}), skipping`); | ||
| return; | ||
@@ -195,8 +379,35 @@ } | ||
| const result = extractInstinct(analysis); | ||
| if (result) { | ||
| log(`Successfully extracted: ${result}`); | ||
| // Apply dynamic daily limit | ||
| const dynamicLimit = getDailyLimit(analysis.user_msg_count); | ||
| if (todayCount >= dynamicLimit) { | ||
| log(`Dynamic limit reached (${todayCount}/${dynamicLimit}), skipping`); | ||
| return; | ||
| } | ||
| const existing = getExistingInstincts(); | ||
| log(`Analyzing: ${analysis.total_observations} observations, ${analysis.frequent_tools.length} patterns (limit: ${dynamicLimit})`); | ||
| const result = extractFromSession(analysis); | ||
| const stats = processResult(result, { existing, todayCount, dynamicLimit }); | ||
| if (stats.instincts > 0) { | ||
| updateCooldown(todayCount + stats.instincts); | ||
| } | ||
| // Update scan state | ||
| try { | ||
| const obsModified = fs.existsSync(OBS_FILE) ? fs.statSync(OBS_FILE).mtimeMs : 0; | ||
| const tmp = SCAN_STATE_FILE + '.tmp'; | ||
| fs.writeFileSync(tmp, JSON.stringify({ | ||
| last_scan_time: new Date().toISOString(), | ||
| last_obs_mtime: obsModified, | ||
| last_stats: stats | ||
| }, null, 2) + '\n'); | ||
| fs.renameSync(tmp, SCAN_STATE_FILE); | ||
| } catch {} | ||
| log(`Done: ${stats.instincts} instincts, ${stats.memory} memory suggestions, ${stats.research} research topics`); | ||
| } | ||
| main(); |
+57
-17
@@ -5,2 +5,4 @@ #!/usr/bin/env bash | ||
| # Called by Claude Code PostToolUse hook | ||
| # | ||
| # v0.9.0: Added noise filtering, reference tracking, skill name enrichment | ||
@@ -12,4 +14,4 @@ set -euo pipefail | ||
| OBS_FILE="${HOMUNCULUS_DIR}/observations.jsonl" | ||
| REF_FILE="${HOMUNCULUS_DIR}/reference-tracking.jsonl" | ||
| MAX_SIZE=$((10 * 1024 * 1024)) # 10MB | ||
| COUNTER_FILE="/tmp/homunculus-tool-count-$$" | ||
@@ -31,24 +33,62 @@ PHASE="${1:-unknown}" | ||
| # Only observe post-tool usage | ||
| TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ") | ||
| # ── Noise filter ── | ||
| # Skip low-value observations to keep observations.jsonl meaningful. | ||
| # Read-only tools provide no actionable patterns for evolution. | ||
| # Exception: Read calls targeting instinct/skill files are tracked for reference analysis. | ||
| case "$TOOL_NAME" in | ||
| Read) | ||
| # Track instinct/skill reads for reference analysis (used by prune-instincts.js) | ||
| if command -v jq &>/dev/null && [ -n "$INPUT" ]; then | ||
| FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty' 2>/dev/null) | ||
| case "$FILE_PATH" in | ||
| */instincts/personal/*|*/instincts/archived/*|*/evolved/skills/*|*/evolved/agents/*) | ||
| mkdir -p "$(dirname "$REF_FILE")" | ||
| echo "$INPUT" | jq -c --arg ts "$TIMESTAMP" --arg path "$FILE_PATH" \ | ||
| '{timestamp: $ts, type: "reference", path: $path}' \ | ||
| >> "$REF_FILE" 2>/dev/null || true | ||
| ;; | ||
| esac | ||
| fi | ||
| exit 0 | ||
| ;; | ||
| # Skip entirely: read-only tools provide no actionable patterns | ||
| Glob|Grep|TodoWrite|TodoRead) | ||
| exit 0 | ||
| ;; | ||
| Skill) | ||
| # Enrich tool name with specific skill for evolution stats | ||
| if command -v jq &>/dev/null && [ -n "$INPUT" ]; then | ||
| SKILL_NAME=$(echo "$INPUT" | jq -r '.tool_input.skill // empty' 2>/dev/null) | ||
| [ -n "$SKILL_NAME" ] && TOOL_NAME="Skill:${SKILL_NAME}" | ||
| fi | ||
| ;; | ||
| # Skip pre-phase: we only care about results (post) for write operations | ||
| Bash|Write|Edit|NotebookEdit) | ||
| [ "$PHASE" = "pre" ] && exit 0 | ||
| ;; | ||
| esac | ||
| # Only observe post-tool usage (after noise filter, so Skill enrichment runs on post) | ||
| [ "$PHASE" != "post" ] && exit 0 | ||
| # Rotate if too large | ||
| if [ -f "$OBS_FILE" ] && [ "$(stat -f%z "$OBS_FILE" 2>/dev/null || stat -c%s "$OBS_FILE" 2>/dev/null || echo 0)" -gt "$MAX_SIZE" ]; then | ||
| TIMESTAMP=$(date +%Y%m%d%H%M%S) | ||
| gzip -c "$OBS_FILE" > "${OBS_FILE}.${TIMESTAMP}.gz" | ||
| : > "$OBS_FILE" | ||
| fi | ||
| # Write observation | ||
| TIMESTAMP=$(date -u +%Y-%m-%dT%H:%M:%SZ) | ||
| # ── Write observation ── | ||
| mkdir -p "$(dirname "$OBS_FILE")" | ||
| if command -v jq &>/dev/null; then | ||
| jq -nc \ | ||
| --arg ts "$TIMESTAMP" \ | ||
| --arg phase "$PHASE" \ | ||
| --arg tool "$TOOL_NAME" \ | ||
| '{timestamp: $ts, phase: $phase, tool: $tool}' >> "$OBS_FILE" | ||
| if command -v jq &>/dev/null && [ -n "$INPUT" ]; then | ||
| # Include agent context for pattern analysis | ||
| echo "$INPUT" | jq -c --arg ts "$TIMESTAMP" --arg phase "$PHASE" --arg tool "$TOOL_NAME" \ | ||
| '{timestamp: $ts, phase: $phase, tool: $tool, | ||
| agent_id: (.agent_id // null), agent_type: (.agent_type // null)}' \ | ||
| >> "$OBS_FILE" 2>/dev/null || true | ||
| else | ||
| echo "{\"timestamp\":\"$TIMESTAMP\",\"phase\":\"$PHASE\",\"tool\":\"$TOOL_NAME\"}" >> "$OBS_FILE" | ||
| fi | ||
| # ── Auto-archive if too large ── | ||
| if [ -f "$OBS_FILE" ] && [ "$(stat -f%z "$OBS_FILE" 2>/dev/null || stat -c%s "$OBS_FILE" 2>/dev/null || echo 0)" -gt "$MAX_SIZE" ]; then | ||
| ARCHIVE="$OBS_FILE.$(date +%Y%m%d%H%M%S).gz" | ||
| gzip -c "$OBS_FILE" > "$ARCHIVE" | ||
| : > "$OBS_FILE" | ||
| fi |
+217
-76
| #!/usr/bin/env node | ||
| // prune-instincts.js — Auto-archive low-value instincts | ||
| // | ||
| // v0.9.0: Reference frequency scoring, 3-tier skill coverage, grace period, --json | ||
| // | ||
| // Scoring dimensions: | ||
| // 0. Capacity pressure (soft limit) | ||
| // 1. Age (older = more likely outdated) | ||
| // 2. Confidence with time decay (half-life based) | ||
| // 3. Skill coverage (already covered by a skill = lower value) | ||
| // 2. Reference frequency (actually used = higher value) | ||
| // 3. Skill coverage — 3 tiers: direct evolved / high overlap / partial overlap | ||
| // 4. Content size (tiny instincts = less valuable) | ||
| // 5. Confidence with time decay (half-life based, 14-day grace period) | ||
| // | ||
@@ -13,2 +18,3 @@ // Usage: | ||
| // node prune-instincts.js --threshold 40 # Custom score threshold (default: 75) | ||
| // node prune-instincts.js --json # Output JSON for programmatic use | ||
@@ -20,11 +26,14 @@ const fs = require('fs'); | ||
| const BASE_DIR = process.env.HOMUNCULUS_BASE || process.cwd(); | ||
| const PERSONAL_DIR = path.join(BASE_DIR, 'homunculus', 'instincts', 'personal'); | ||
| const ARCHIVED_DIR = path.join(BASE_DIR, 'homunculus', 'instincts', 'archived'); | ||
| const SKILLS_DIR = path.join(BASE_DIR, 'homunculus', 'evolved', 'skills'); | ||
| const HOMUNCULUS_DIR = path.join(BASE_DIR, 'homunculus'); | ||
| const PERSONAL_DIR = path.join(HOMUNCULUS_DIR, 'instincts', 'personal'); | ||
| const ARCHIVED_DIR = path.join(HOMUNCULUS_DIR, 'instincts', 'archived'); | ||
| const SKILLS_DIR = path.join(HOMUNCULUS_DIR, 'evolved', 'skills'); | ||
| const REF_FILE = path.join(HOMUNCULUS_DIR, 'reference-tracking.jsonl'); | ||
| const args = process.argv.slice(2); | ||
| const applyMode = args.includes('--apply'); | ||
| const jsonMode = args.includes('--json'); | ||
| const thresholdIdx = args.indexOf('--threshold'); | ||
| const ARCHIVE_THRESHOLD = thresholdIdx >= 0 ? parseInt(args[thresholdIdx + 1], 10) : 75; | ||
| const CAPACITY_SOFT_LIMIT = 80; | ||
| const CAPACITY_SOFT_LIMIT = parseInt(process.env.HOMUNCULUS_CAPACITY_LIMIT || '80'); | ||
@@ -34,2 +43,3 @@ // Confidence decay: half-life in days | ||
| const DECAY_LAMBDA = Math.LN2 / CONFIDENCE_HALF_LIFE_DAYS; | ||
| const GRACE_PERIOD_DAYS = 14; // No decay for first 2 weeks | ||
@@ -44,3 +54,4 @@ function safeRead(fp) { | ||
| const confidence = parseFloat((content.match(/confidence:\s*([\d.]+)/im) || [])[1] || '0.5'); | ||
| // Support both plain and markdown-bold metadata formats | ||
| const confidence = parseFloat((content.match(/\*?\*?confidence\*?\*?:?\s*([\d.]+)/im) || [])[1] || '0.5'); | ||
| const createdMatch = content.match(/(?:extracted|created|date):\s*"?([^"\n]+)"?/im); | ||
@@ -50,63 +61,146 @@ const created = createdMatch ? new Date(createdMatch[1]) : null; | ||
| const updated = updatedMatch ? new Date(updatedMatch[1]) : created; | ||
| const source = (content.match(/\*?\*?source\*?\*?:?\s*"?([^"\n]+)"?/im) || [])[1]?.trim() || 'unknown'; | ||
| // Confidence decay | ||
| const daysSinceUpdate = updated ? (Date.now() - updated.getTime()) / 86400000 : 180; | ||
| const effectiveConfidence = confidence * Math.exp(-DECAY_LAMBDA * daysSinceUpdate); | ||
| return { name, filepath, content, confidence, created, updated, source }; | ||
| } | ||
| return { name, confidence, effectiveConfidence, created, updated, daysSinceUpdate, content }; | ||
| // Load reference tracking data (from observe.sh) | ||
| function loadReferenceData() { | ||
| const refs = {}; | ||
| const content = safeRead(REF_FILE); | ||
| if (!content) return refs; | ||
| for (const line of content.split('\n').filter(l => l.trim())) { | ||
| try { | ||
| const entry = JSON.parse(line); | ||
| const p = entry.path || ''; | ||
| if (!refs[p]) refs[p] = { count: 0, lastRef: null }; | ||
| refs[p].count++; | ||
| const ts = entry.timestamp ? new Date(entry.timestamp).getTime() : 0; | ||
| if (!refs[p].lastRef || ts > refs[p].lastRef) refs[p].lastRef = ts; | ||
| } catch {} | ||
| } | ||
| return refs; | ||
| } | ||
| function getSkillCoverage() { | ||
| const coverage = new Set(); | ||
| if (!fs.existsSync(SKILLS_DIR)) return coverage; | ||
| for (const file of fs.readdirSync(SKILLS_DIR).filter(f => f.endsWith('.md'))) { | ||
| const content = safeRead(path.join(SKILLS_DIR, file)); | ||
| // Skills often reference instinct names they were derived from | ||
| const sourceMatch = content.match(/source_instincts?:(.+)/gim); | ||
| if (sourceMatch) { | ||
| for (const match of sourceMatch) { | ||
| const names = match.replace(/source_instincts?:/i, '').split(/[,\s]+/); | ||
| names.forEach(n => coverage.add(n.trim())); | ||
| // Build token sets + evolved-from lists for each skill (3-tier coverage detection) | ||
| function loadSkillData() { | ||
| const skills = {}; | ||
| if (!fs.existsSync(SKILLS_DIR)) return skills; | ||
| try { | ||
| const entries = fs.readdirSync(SKILLS_DIR, { withFileTypes: true }); | ||
| for (const entry of entries) { | ||
| let content; | ||
| if (entry.isFile() && entry.name.endsWith('.md')) { | ||
| content = safeRead(path.join(SKILLS_DIR, entry.name)).toLowerCase(); | ||
| } else if (entry.isDirectory()) { | ||
| const indexPath = path.join(SKILLS_DIR, entry.name, 'index.md'); | ||
| if (fs.existsSync(indexPath)) content = safeRead(indexPath).toLowerCase(); | ||
| } | ||
| if (!content) continue; | ||
| const skillName = entry.name.replace('.md', ''); | ||
| const tokens = new Set(content.split(/[\s\-_,./:()\[\]{}#*>|]+/).filter(w => w.length > 4)); | ||
| // Extract "evolved from" instinct names | ||
| const evolvedFrom = new Set(); | ||
| const matches = content.matchAll(/`([^`]+\.md)`/g); | ||
| for (const m of matches) evolvedFrom.add(m[1].replace('.md', '')); | ||
| skills[skillName] = { tokens, evolvedFrom }; | ||
| } | ||
| } | ||
| return coverage; | ||
| } catch {} | ||
| return skills; | ||
| } | ||
| function scoreInstinct(instinct, skillCoverage, totalCount) { | ||
| function scoreInstinct(instinct, refs, skillData, now, totalRefs, activeCount) { | ||
| let score = 100; | ||
| const reasons = []; | ||
| // Effective confidence (with decay) | ||
| if (instinct.effectiveConfidence < 0.5) { | ||
| score -= 30; | ||
| reasons.push(`low effective confidence: ${instinct.effectiveConfidence.toFixed(2)}`); | ||
| } else if (instinct.effectiveConfidence < 0.7) { | ||
| // 0. Capacity pressure | ||
| if (activeCount > CAPACITY_SOFT_LIMIT) { | ||
| const pressure = Math.min((activeCount - CAPACITY_SOFT_LIMIT) * 2, 20); | ||
| score -= pressure; | ||
| reasons.push(`capacity ${activeCount}/${CAPACITY_SOFT_LIMIT} (-${pressure})`); | ||
| } | ||
| // 1. Age penalty | ||
| if (instinct.created) { | ||
| const ageDays = (now - instinct.created.getTime()) / 86400000; | ||
| if (ageDays > 30) { | ||
| const penalty = Math.min(Math.floor((ageDays - 30) / 7) * 5, 30); | ||
| if (penalty > 0) { | ||
| score -= penalty; | ||
| reasons.push(`age ${Math.round(ageDays)}d (-${penalty})`); | ||
| } | ||
| } | ||
| } | ||
| // 2. Reference frequency bonus/penalty | ||
| const refData = refs[instinct.filepath]; | ||
| if (refData && refData.count > 0) { | ||
| const bonus = Math.min(refData.count * 5, 25); | ||
| score += bonus; | ||
| reasons.push(`refs ${refData.count} (+${bonus})`); | ||
| } else if (totalRefs > 50) { | ||
| // Only penalize if we have enough data to be confident | ||
| score -= 15; | ||
| reasons.push(`medium confidence: ${instinct.effectiveConfidence.toFixed(2)}`); | ||
| reasons.push('no refs (-15)'); | ||
| } | ||
| // Age penalty | ||
| if (instinct.daysSinceUpdate > 60) { | ||
| score -= 20; | ||
| reasons.push(`old: ${Math.floor(instinct.daysSinceUpdate)} days`); | ||
| } else if (instinct.daysSinceUpdate > 30) { | ||
| score -= 10; | ||
| reasons.push(`aging: ${Math.floor(instinct.daysSinceUpdate)} days`); | ||
| // 3. Skill coverage — 3 tiers | ||
| const instinctContent = (instinct.name + ' ' + instinct.content).toLowerCase(); | ||
| const instTokens = new Set(instinctContent.split(/[\s\-_,./:()\[\]{}#*>|]+/).filter(w => w.length > 4)); | ||
| let bestSkill = null, bestRatio = 0, directlyEvolved = false; | ||
| for (const [skillName, { tokens: skillTokens, evolvedFrom }] of Object.entries(skillData)) { | ||
| if (evolvedFrom.has(instinct.name)) { | ||
| directlyEvolved = true; | ||
| bestSkill = skillName; | ||
| bestRatio = 1.0; | ||
| break; | ||
| } | ||
| let overlap = 0; | ||
| for (const t of instTokens) { if (skillTokens.has(t)) overlap++; } | ||
| const ratio = instTokens.size > 0 ? overlap / instTokens.size : 0; | ||
| if (ratio > bestRatio) { bestRatio = ratio; bestSkill = skillName; } | ||
| } | ||
| // Skill coverage | ||
| if (skillCoverage.has(instinct.name)) { | ||
| score -= 25; | ||
| reasons.push('covered by skill'); | ||
| if (directlyEvolved) { | ||
| score -= 35; | ||
| reasons.push(`directly evolved into ${bestSkill} (-35)`); | ||
| } else if (bestRatio >= 0.45) { | ||
| const penalty = Math.min(Math.round(bestRatio * 50), 35); | ||
| score -= penalty; | ||
| reasons.push(`${Math.round(bestRatio * 100)}% covered by ${bestSkill} (-${penalty})`); | ||
| } else if (bestRatio >= 0.30) { | ||
| const penalty = Math.min(Math.round(bestRatio * 25), 15); | ||
| score -= penalty; | ||
| reasons.push(`${Math.round(bestRatio * 100)}% partial overlap with ${bestSkill} (-${penalty})`); | ||
| } | ||
| // Capacity pressure | ||
| if (totalCount > CAPACITY_SOFT_LIMIT) { | ||
| score -= Math.min(10, totalCount - CAPACITY_SOFT_LIMIT); | ||
| reasons.push(`capacity pressure: ${totalCount}/${CAPACITY_SOFT_LIMIT}`); | ||
| // 4. Content size penalty | ||
| if (instinct.content.length < 300) { | ||
| score -= 10; | ||
| reasons.push(`tiny ${instinct.content.length}c (-10)`); | ||
| } | ||
| return { score: Math.max(0, score), reasons }; | ||
| // 5. Confidence decay (with grace period) | ||
| let effectiveConfidence = instinct.confidence; | ||
| if (instinct.updated) { | ||
| const daysSinceUpdate = (now - instinct.updated.getTime()) / 86400000; | ||
| if (daysSinceUpdate > GRACE_PERIOD_DAYS) { | ||
| const decayFactor = Math.exp(-DECAY_LAMBDA * (daysSinceUpdate - GRACE_PERIOD_DAYS)); | ||
| effectiveConfidence = instinct.confidence * decayFactor; | ||
| } | ||
| } | ||
| if (effectiveConfidence < 0.3) { | ||
| score -= 20; | ||
| reasons.push(`confidence decayed ${instinct.confidence.toFixed(2)}→${effectiveConfidence.toFixed(2)} (-20)`); | ||
| } else if (effectiveConfidence < 0.5) { | ||
| score -= 15; | ||
| reasons.push(`low effective confidence ${effectiveConfidence.toFixed(2)} (-15)`); | ||
| } else if (effectiveConfidence < 0.7) { | ||
| score -= 5; | ||
| reasons.push(`mid effective confidence ${effectiveConfidence.toFixed(2)} (-5)`); | ||
| } | ||
| return { score: Math.max(0, Math.min(100, score)), reasons, effectiveConfidence }; | ||
| } | ||
@@ -120,49 +214,96 @@ | ||
| const now = Date.now(); | ||
| const files = fs.readdirSync(PERSONAL_DIR).filter(f => f.endsWith('.md')); | ||
| const skillCoverage = getSkillCoverage(); | ||
| const candidates = []; | ||
| const activeCount = files.length; | ||
| for (const file of files) { | ||
| const instinct = parseInstinct(path.join(PERSONAL_DIR, file)); | ||
| const { score, reasons } = scoreInstinct(instinct, skillCoverage, files.length); | ||
| console.log(`\n=== Instinct Pruning Analysis ===`); | ||
| console.log(`Active: ${activeCount} | Soft limit: ${CAPACITY_SOFT_LIMIT} | Threshold: ${ARCHIVE_THRESHOLD}`); | ||
| console.log(`Confidence decay: half-life ${CONFIDENCE_HALF_LIFE_DAYS}d, grace ${GRACE_PERIOD_DAYS}d`); | ||
| console.log(`Mode: ${applyMode ? 'APPLY (will archive)' : 'DRY RUN (preview only)'}\n`); | ||
| if (score < ARCHIVE_THRESHOLD) { | ||
| candidates.push({ ...instinct, score, reasons }); | ||
| } | ||
| const refs = loadReferenceData(); | ||
| const totalRefs = Object.values(refs).reduce((a, b) => a + b.count, 0); | ||
| console.log(`Reference tracking: ${totalRefs} total reads`); | ||
| if (totalRefs < 50) { | ||
| console.log(` (Low data — reference scoring will be conservative)\n`); | ||
| } else { | ||
| console.log(); | ||
| } | ||
| candidates.sort((a, b) => a.score - b.score); | ||
| const skillData = loadSkillData(); | ||
| console.log(`Skills loaded: ${Object.keys(skillData).length}\n`); | ||
| console.log(`\nInstincts: ${files.length} active | Threshold: ${ARCHIVE_THRESHOLD} | Candidates: ${candidates.length}`); | ||
| console.log('─'.repeat(60)); | ||
| // Score all instincts | ||
| const scored = files.map(f => { | ||
| const instinct = parseInstinct(path.join(PERSONAL_DIR, f)); | ||
| const { score, reasons, effectiveConfidence } = scoreInstinct(instinct, refs, skillData, now, totalRefs, activeCount); | ||
| return { ...instinct, score, reasons, effectiveConfidence }; | ||
| }).sort((a, b) => a.score - b.score); | ||
| if (candidates.length === 0) { | ||
| console.log('No archive candidates found.'); | ||
| return; | ||
| const candidates = scored.filter(i => i.score < ARCHIVE_THRESHOLD); | ||
| const atRisk = scored.filter(i => i.score >= ARCHIVE_THRESHOLD && i.score < 50); | ||
| // Display candidates | ||
| if (candidates.length > 0) { | ||
| console.log(`--- Archive Candidates (score < ${ARCHIVE_THRESHOLD}) ---\n`); | ||
| for (const c of candidates) { | ||
| console.log(` [${c.score.toString().padStart(3)}] ${c.name}`); | ||
| console.log(` ${c.reasons.join(' | ')}`); | ||
| } | ||
| console.log(); | ||
| } else { | ||
| console.log(`No archive candidates (all scores >= ${ARCHIVE_THRESHOLD})\n`); | ||
| } | ||
| for (const c of candidates) { | ||
| console.log(` ${c.score.toString().padStart(3)} | ${c.name}`); | ||
| console.log(` ${c.reasons.join(', ')}`); | ||
| // Display at-risk | ||
| if (atRisk.length > 0) { | ||
| console.log(`--- At Risk (${ARCHIVE_THRESHOLD}–49) ---\n`); | ||
| for (const c of atRisk) { | ||
| console.log(` [${c.score.toString().padStart(3)}] ${c.name}`); | ||
| console.log(` ${c.reasons.join(' | ')}`); | ||
| } | ||
| console.log(); | ||
| } | ||
| if (applyMode) { | ||
| console.log('\nArchiving...'); | ||
| // Summary | ||
| const afterArchive = activeCount - candidates.length; | ||
| console.log(`--- Summary ---`); | ||
| console.log(`Would archive: ${candidates.length} instincts`); | ||
| console.log(`After pruning: ${afterArchive} active (${afterArchive > CAPACITY_SOFT_LIMIT ? 'above soft limit' : 'within capacity'})`); | ||
| // Apply | ||
| if (applyMode && candidates.length > 0) { | ||
| console.log(`\nArchiving ${candidates.length} instincts...\n`); | ||
| if (!fs.existsSync(ARCHIVED_DIR)) fs.mkdirSync(ARCHIVED_DIR, { recursive: true }); | ||
| for (const c of candidates) { | ||
| const src = path.join(PERSONAL_DIR, `${c.name}.md`); | ||
| const dest = path.join(ARCHIVED_DIR, `${c.name}.md`); | ||
| // Append archive metadata before moving | ||
| const note = `\n\n---\n_Archived: ${new Date().toISOString().slice(0, 10)} | Covered-by: ${c.reason || 'low-score'} | Score: ${c.score}_\n`; | ||
| fs.appendFileSync(src, note); | ||
| fs.renameSync(src, dest); | ||
| console.log(` Archived: ${c.name}`); | ||
| const src = c.filepath; | ||
| const dst = path.join(ARCHIVED_DIR, path.basename(src)); | ||
| const coveredBy = c.reasons | ||
| .filter(r => r.includes('evolved into') || r.includes('covered by') || r.includes('overlap with')) | ||
| .map(r => r.replace(/\s*\(-\d+\)/, '')) | ||
| .join('; ') || 'low score'; | ||
| const note = `\n\n---\n_Archived: ${new Date().toISOString().slice(0, 10)} | Score: ${c.score} | Covered-by: ${coveredBy}_\n`; | ||
| const content = safeRead(src) + note; | ||
| fs.writeFileSync(dst, content); | ||
| fs.unlinkSync(src); | ||
| console.log(` Archived: ${c.name} (score: ${c.score})`); | ||
| } | ||
| console.log(`\nDone. Archived ${candidates.length} instincts.`); | ||
| } else { | ||
| console.log(`\nDone. ${candidates.length} instincts archived.`); | ||
| } else if (!applyMode && candidates.length > 0) { | ||
| console.log(`\nDry run. Use --apply to archive these ${candidates.length} instincts.`); | ||
| } | ||
| // JSON output | ||
| if (jsonMode) { | ||
| const output = { | ||
| active: activeCount, | ||
| candidates: candidates.map(c => ({ name: c.name, score: c.score, reasons: c.reasons })), | ||
| at_risk: atRisk.map(c => ({ name: c.name, score: c.score, reasons: c.reasons })), | ||
| after_pruning: afterArchive | ||
| }; | ||
| console.log('\n' + JSON.stringify(output, null, 2)); | ||
| } | ||
| } | ||
| main(); |
@@ -38,3 +38,3 @@ # Nightly Agent | ||
| | **P0** | Complete assigned high-priority tasks | | ||
| | **P1** | Evolution cycle — route instincts to 8 mechanisms (hook/rule/skill/script/agent/...), eval + improve skills, review workflow/subagent health, check all mechanisms working, review goal implementations | | ||
| | **P1** | Evolution cycle — extract instincts + memory suggestions + research topics, route instincts to 8 mechanisms (hook/rule/skill/script/agent/...), eval + improve skills, review goal implementations | | ||
| | **P2** | Research — scan tech news, changelogs, community (with cross-night dedup) | | ||
@@ -117,2 +117,13 @@ | **P3** | Experiments — generate hypotheses from weak goals, test in isolated worktrees | | ||
| ## Reports Directory | ||
| The evolution engine writes suggestions to `homunculus/reports/` (non-invasive — you decide what to adopt): | ||
| | File | Purpose | | ||
| |------|---------| | ||
| | `memory-suggestions.jsonl` | Context worth preserving (user preferences, project decisions, external resources) | | ||
| | `research-queue.jsonl` | Topics mentioned but not explored — candidates for nightly research | | ||
| Review these during `/hm-night` or check them manually anytime. | ||
| ## Morning Report | ||
@@ -123,2 +134,3 @@ | ||
| - System evolution (instinct routing, skill evals) | ||
| - Memory suggestions extracted from sessions | ||
| - Research topics (with source URLs and goal relevance tags) | ||
@@ -125,0 +137,0 @@ - Experiments (hypothesis, result, merged or discarded) |
+1
-1
| { | ||
| "name": "homunculus-code", | ||
| "version": "0.8.0", | ||
| "version": "0.9.0", | ||
| "description": "A self-evolving AI assistant that grows smarter every night", | ||
@@ -5,0 +5,0 @@ "bin": { |
+11
-0
@@ -167,2 +167,13 @@ # Homunculus for Claude Code | ||
| ### v0.9.0 — Evolution Engine Upgrade (Mar 2026) | ||
| - **Smart observation** — `observe.sh` now filters noise (skips Read/Glob/Grep, only records post-phase for writes) and tracks reference frequency — which instincts and skills are actually being read | ||
| - **Three-layer extraction** — `evaluate-session.js` now extracts instincts + memory suggestions + research topics in a single pass. Memory and research are written to `homunculus/reports/` for you to review (non-invasive) | ||
| - **Semantic dedup** — New instincts can declare `supersedes` to auto-archive older instincts they replace | ||
| - **Write Gate** — Extraction prompt now includes quality criteria (must change future behavior, capture a commitment, or preserve a decision rationale) | ||
| - **Dynamic daily cap** — Instinct extraction limit scales with session size (1 for short sessions, up to 5 for long ones) | ||
| - **Smarter pruning** — `prune-instincts.js` now uses reference frequency scoring (+25 for frequently used, -15 for never referenced), 3-tier skill coverage detection (direct evolved / high overlap / partial), 14-day grace period before confidence decay, and at-risk warnings | ||
| - **Idempotency** — Extraction tracks scan state to avoid reprocessing the same observations | ||
| - **`--json` output** — `prune-instincts.js` supports `--json` for programmatic use | ||
| ### v0.8.0 — Upgrade Mechanism (Mar 2026) | ||
@@ -169,0 +180,0 @@ |
@@ -14,11 +14,13 @@ # Evolution System (Homunculus) | ||
| ``` | ||
| Observe (observations.jsonl) → Pattern detection (confidence > 0.7) → instincts/personal/*.md | ||
| ↓ | ||
| /evolve → evolved/skills/*.md | ||
| ↓ | ||
| eval → improve loop → 100% pass | ||
| Observe (observations.jsonl) → Three-layer extraction: | ||
| ├── Instincts (behavioral patterns, confidence > 0.7, Write Gate) | ||
| │ ↓ supersedes check → auto-archive replaced instincts | ||
| │ ↓ /evolve → route to best mechanism (hook/rule/skill/script/agent/...) | ||
| │ ↓ if skill → eval → improve loop → 100% pass | ||
| ├── Memory suggestions → reports/memory-suggestions.jsonl (user reviews) | ||
| └── Research topics → reports/research-queue.jsonl (nightly agent picks up) | ||
| ``` | ||
| ## Manual Triggers | ||
| - `/evolve` — Aggregate instincts into a skill | ||
| - `/evolve` — Route instincts to the best mechanism (skill is just one of 8) | ||
| - `/eval-skill` — Test a skill against its eval spec | ||
@@ -28,4 +30,5 @@ - `/improve-skill` — Auto-improve until eval passes | ||
| ## Automatic Maintenance | ||
| - Instinct extraction from session observations | ||
| - Instinct pruning (confidence decay, skill coverage check) | ||
| - Three-layer extraction: instincts + memory + research from sessions | ||
| - Semantic dedup: new instincts auto-archive superseded ones | ||
| - Smart pruning: reference frequency + 3-tier skill coverage + confidence decay (14-day grace) | ||
| - Skill eval → improve pipeline |
Environment variable access
Supply chain riskPackage accesses environment variables, which may be a sign of credential stuffing or data theft.
Found 3 instances
AI-detected potential code anomaly
Supply chain riskAI has identified unusual behaviors that may pose a security risk.
Long strings
Supply chain riskContains long string literals, which may be a sign of obfuscated or packed code.
URL strings
Supply chain riskPackage contains fragments of external URLs or IP addresses, which the package may be accessing at runtime.
Environment variable access
Supply chain riskPackage accesses environment variables, which may be a sign of credential stuffing or data theft.
AI-detected potential code anomaly
Supply chain riskAI has identified unusual behaviors that may pose a security risk.
URL strings
Supply chain riskPackage contains fragments of external URLs or IP addresses, which the package may be accessing at runtime.
460881
4.55%62
1.64%1382
29.52%650
1.72%17
21.43%