🎩 You're Invited:Meet the Socket team at Black Hat in Las Vegas, August 3-6.RSVP
Sign In

homunculus-code

Package Overview
Dependencies
Maintainers
1
Versions
24
Alerts
File Explorer

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

homunculus-code - npm Package Compare versions

Comparing version
0.8.0
to
0.9.0
+29
CLAUDE.md
# Homunculus — Development Guide
## Release Checklist
Every change to this repo must complete these steps in order:
1. **Code changes** — implement the feature/fix
2. **README.md** — update What's New section + any affected docs
3. **package.json** — bump version (semver: patch/minor/major)
4. **git commit + push** — descriptive commit message
5. **npm publish** — `npm publish` from repo root
6. **GitHub release** — `gh release create vX.Y.Z` with release notes
7. **assistant plan.md** — sync `~/assistant/projects/homunculus/plan.md` if applicable
## Project Structure
- `bin/` — CLI entry points (init, upgrade, night)
- `core/` — Evolution scripts copied to user projects during init
- `commands/` — Slash commands copied to `.claude/commands/` during init
- `templates/` — CLAUDE.md, architecture.yaml, rules templates for init
- `docs/` — Long-form documentation
- `examples/reference/` — Real-world reference implementation from the author's system
## Key Design Decisions
- **Zero dependencies** — Only Node.js stdlib + Claude CLI
- **File copy, not symlink** — init copies files so users can customize freely
- **Manifest-based upgrade** — `.manifest.json` tracks SHA256 hashes for diff-aware upgrades
- **Never touch user content** — upgrade never modifies CLAUDE.md, architecture.yaml, instincts, skills
+11
-9

@@ -108,16 +108,18 @@ # Architecture: Goal-Tree-Driven Evolution

```
Tool usage → observe.sh (hook) → observations.jsonl
|
evaluate-session.js (SessionEnd)
|
v
instincts/personal/*.md
(confidence-scored patterns)
(tagged: suggested_mechanism + goal_path)
Tool usage → observe.sh (hook, noise-filtered) → observations.jsonl
| |
|→ reference-tracking.jsonl evaluate-session.js
(which instincts/skills |
are actually read) v
┌── instincts/personal/*.md
│ (confidence-scored, mechanism-tagged)
├── reports/memory-suggestions.jsonl
└── reports/research-queue.jsonl
```
Instincts are small behavioral patterns with:
- **Confidence scores** — increase with reinforcement, decay over time
- **Confidence scores** — increase with reinforcement, decay over time (14-day grace period)
- **Mechanism tag** — which implementation type fits best (hook/rule/skill/script/...)
- **Goal path** — which goal in the tree this serves
- **Semantic dedup** — new instincts can `supersede` older ones, auto-archiving them
- **Automatic archival** — archived once implemented by any mechanism

@@ -124,0 +126,0 @@

#!/usr/bin/env node
// evaluate-session.js — Extract instincts from session observations
// evaluate-session.js — Extract instincts, memory suggestions, and research topics from sessions
// Part of the Homunculus evolution pipeline
//
// v0.9.0: Three-layer extraction, dynamic daily cap, semantic dedup, Write Gate
//
// Flow:
// 1. Read observations.jsonl, analyze tool usage patterns
// 2. Determine if session is worth extracting (enough activity)
// 3. Use Claude CLI to extract behavioral patterns as instincts
// 1. Read observations.jsonl OR session transcript, analyze tool usage patterns
// 2. Check daily cooldown + dynamic limit based on session size
// 3. Use Claude CLI to extract behavioral patterns as instincts + memory + research
// 4. Write instincts to homunculus/instincts/personal/
// 5. Write memory/research suggestions to homunculus/reports/

@@ -19,9 +22,16 @@ const fs = require('fs');

const INSTINCTS_DIR = path.join(HOMUNCULUS_DIR, 'instincts', 'personal');
const ARCHIVE_DIR = path.join(HOMUNCULUS_DIR, 'instincts', 'archived');
const OBS_FILE = path.join(HOMUNCULUS_DIR, 'observations.jsonl');
const REPORTS_DIR = path.join(HOMUNCULUS_DIR, 'reports');
const COOLDOWN_FILE = path.join(HOMUNCULUS_DIR, '.cooldown.json');
const SCAN_STATE_FILE = path.join(HOMUNCULUS_DIR, '.scan-state.json');
// Extraction thresholds (auto-tunable)
// Extraction thresholds
const CONFIG = {
min_messages: parseInt(process.env.HOMUNCULUS_MIN_MESSAGES || '10'),
min_tool_repeats: parseInt(process.env.HOMUNCULUS_MIN_TOOL_REPEATS || '5'),
daily_limit: parseInt(process.env.HOMUNCULUS_DAILY_LIMIT || '3')
// Dynamic daily cap: scales with session size
daily_limit_thresholds: [0, 30, 60, 100],
daily_limit_values: [1, 2, 3, 5],
hard_cap: 5
};

@@ -41,17 +51,87 @@

// Count today's extractions to respect daily limit
function getTodayExtractions() {
const today = getDateString();
// Dynamic daily limit based on session size
function getDailyLimit(msgCount) {
const { daily_limit_thresholds: thresholds, daily_limit_values: limits } = CONFIG;
for (let i = thresholds.length - 1; i >= 0; i--) {
if (msgCount >= thresholds[i]) return limits[i] || 1;
}
return 1;
}
// Cooldown state file (O(1) lookup instead of scanning all instinct files)
function getCooldown() {
try {
const files = fs.readdirSync(INSTINCTS_DIR);
return files.filter(f => {
const content = fs.readFileSync(path.join(INSTINCTS_DIR, f), 'utf8');
return content.includes(today);
}).length;
return JSON.parse(fs.readFileSync(COOLDOWN_FILE, 'utf8'));
} catch {
return 0;
return { date: null, count: 0 };
}
}
// Analyze observations to find patterns
function updateCooldown(count) {
const today = getDateString();
fs.writeFileSync(COOLDOWN_FILE, JSON.stringify({ date: today, count }));
}
// Get existing instinct names (for duplicate check)
function getExistingInstincts() {
ensureDir(INSTINCTS_DIR);
return fs.readdirSync(INSTINCTS_DIR)
.filter(f => f.endsWith('.md'))
.map(f => f.replace(/\.md$/, ''));
}
// Get existing instincts with titles for semantic supersede detection
function getExistingInstinctsWithTitles() {
ensureDir(INSTINCTS_DIR);
const results = [];
for (const f of fs.readdirSync(INSTINCTS_DIR).filter(f => f.endsWith('.md'))) {
const name = f.replace(/\.md$/, '');
try {
const content = fs.readFileSync(path.join(INSTINCTS_DIR, f), 'utf8');
const titleMatch = content.match(/^# Instinct: (.+)/m);
const title = titleMatch ? titleMatch[1].trim() : name;
results.push(`${name}: ${title}`);
} catch { results.push(name); }
}
return results;
}
// Archive an instinct that has been superseded by a newer one
function archiveSupersededInstinct(name, supersededBy) {
ensureDir(ARCHIVE_DIR);
const src = path.join(INSTINCTS_DIR, `${name}.md`);
const dst = path.join(ARCHIVE_DIR, `${name}.md`);
if (!fs.existsSync(src)) return false;
let content = fs.readFileSync(src, 'utf8');
content += `\n\n---\n**Archived:** ${new Date().toISOString()}\n**Covered-by:** ${supersededBy} (supersede)\n`;
fs.writeFileSync(dst, content);
fs.unlinkSync(src);
return true;
}
// Save memory suggestion to reports (non-invasive — user decides what to adopt)
function saveMemorySuggestion(entry) {
ensureDir(REPORTS_DIR);
const file = path.join(REPORTS_DIR, 'memory-suggestions.jsonl');
const record = {
...entry,
timestamp: new Date().toISOString(),
status: 'pending'
};
fs.appendFileSync(file, JSON.stringify(record) + '\n');
}
// Save research topic to reports
function saveResearchTopic(entry) {
ensureDir(REPORTS_DIR);
const file = path.join(REPORTS_DIR, 'research-queue.jsonl');
const record = {
...entry,
timestamp: new Date().toISOString(),
status: 'pending'
};
fs.appendFileSync(file, JSON.stringify(record) + '\n');
}
// Analyze observations.jsonl for tool patterns
function analyzeObservations() {

@@ -69,3 +149,2 @@ if (!fs.existsSync(OBS_FILE)) {

// Count tool usage
const toolCounts = {};

@@ -80,3 +159,2 @@ for (const line of lines) {

// Find high-frequency tools
const frequentTools = Object.entries(toolCounts)

@@ -94,85 +172,101 @@ .filter(([_, count]) => count >= CONFIG.min_tool_repeats)

frequent_tools: frequentTools,
tool_counts: toolCounts
tool_counts: toolCounts,
user_msg_count: lines.length // approximate
};
}
// Extract instinct using Claude CLI
function extractInstinct(analysis) {
const toolSummary = analysis.frequent_tools
.map(([tool, count]) => `${tool}: ${count} times`)
.join(', ');
// Read architecture.yaml goal names for goal_path tagging
let goalHint = '';
// Read goal tree names for goal_path tagging
function getGoalHint() {
try {
const archFile = path.join(BASE_DIR, 'architecture.yaml');
if (fs.existsSync(archFile)) {
const archContent = fs.readFileSync(archFile, 'utf8');
const goals = [...archContent.matchAll(/^ (\w+):\s*$/gm)].map(m => m[1]).filter(g => !['goals', 'metrics', 'agents', 'health_check', 'test_config'].includes(g));
goalHint = goals.length > 0 ? `\nGoal tree top-level goals: ${goals.join(', ')}` : '';
const content = fs.readFileSync(archFile, 'utf8');
const goals = [...content.matchAll(/^ (\w+):\s*$/gm)]
.map(m => m[1])
.filter(g => !['goals', 'metrics', 'agents', 'health_check', 'test_config'].includes(g));
return goals.length > 0 ? goals.join(', ') : '';
}
} catch {}
return '';
}
const prompt = `Based on this session's tool usage patterns, extract ONE behavioral instinct that would be useful to remember for future sessions.
// Extract instincts + memory + research using Claude CLI
function extractFromSession(analysis) {
const toolSummary = analysis.frequent_tools
.map(([tool, count]) => `${tool}: ${count} times`)
.join(', ');
Tool usage: ${toolSummary}
Total observations: ${analysis.total_observations}
${goalHint}
const goalHint = getGoalHint();
const existingWithTitles = getExistingInstinctsWithTitles();
Implementation mechanisms (pick the best fit for suggested_mechanism):
- hook: deterministic, every time, zero AI judgment (e.g. lint after edit)
- rule: path-scoped guidance for specific directories
- skill: reusable knowledge collection with eval spec
- script: periodic automation, no AI needed at runtime
- agent: isolated context, specialist role
- system: infrastructure-level
const prompt = `Analyze this Claude Code session and extract reusable behavioral patterns.
Write the instinct as a markdown file with this format:
## Session Stats
- Observations: ${analysis.total_observations}
- Frequent tools: ${toolSummary}
${goalHint ? `\n## Goal Tree (for goal_path)\n${goalHint}` : ''}
---
name: descriptive-kebab-case-name
category: one of [coding, debugging, workflow, communication, tool-usage]
confidence: 0.7
extracted: "${getDateString()}"
source: "session observation"
suggested_mechanism: hook|rule|skill|script|agent|system
goal_path: "top_level_goal.sub_goal or unrooted if no match"
---
## Existing Instincts (avoid duplicates)
${existingWithTitles.length > 0 ? existingWithTitles.join('\n') : '(none yet)'}
## Pattern
[Describe the behavioral pattern in 1-2 sentences]
## Write Gate (must pass at least one to record)
An observation must meet at least one criterion to become an instinct:
- **Changes future behavior**: Would this pattern make future sessions different?
- **Commitment**: User explicitly stated a preference ("always use CLI not MCP")
- **Decision rationale**: Why A not B (key reasoning worth preserving)
- **Stable fact**: System knowledge unlikely to change
- **Explicit instruction**: User said "remember this" or "always do this"
## When to Apply
[When should this pattern be used]
Observations that fail all criteria → no instinct (output empty line).
## Anti-Patterns
[What to avoid]
## Implementation Mechanisms (for suggested_mechanism)
hook (deterministic, every time) | rule (path-scoped guidance) | skill (reusable knowledge with eval) | script (periodic automation) | agent (isolated context specialist) | system (infrastructure)
IMPORTANT: Only output the markdown content, nothing else.`;
## Output Format
try {
const result = execSync(
`claude -p "${prompt.replace(/"/g, '\\"')}" --model claude-sonnet-4-6 --max-budget-usd 0.50`,
{ encoding: 'utf8', timeout: 30000 }
);
### A. Instincts (behavioral patterns)
Find 1-2 reusable patterns (confidence > 0.7, must pass Write Gate):
{"type": "instinct", "name": "kebab-case-name", "confidence": 0.8, "title": "Pattern title", "content": "Full instinct description", "suggested_mechanism": "hook|rule|skill|script|agent|system", "goal_path": "top_goal.sub_goal or unrooted", "supersedes": []}
// Extract name from frontmatter
const nameMatch = result.match(/name:\s*(.+)/);
if (!nameMatch) return null;
- supersedes: if this new pattern semantically replaces an existing instinct, list its name. Leave [] if unsure.
const name = nameMatch[1].trim().replace(/[^a-z0-9-]/g, '-');
const filename = `${name}.md`;
const filepath = path.join(INSTINCTS_DIR, filename);
### B. Memory Suggestions (important context to preserve)
Check if the session contains information worth long-term storage (not instincts, but facts):
- User preferences or corrections
- Project decisions or direction changes
- External resources discovered
- User background updates
if (fs.existsSync(filepath)) {
log(`Instinct ${filename} already exists, skipping`);
return null;
}
{"type": "memory", "category": "user|feedback|project|reference", "title": "Short title", "content": "Content to save"}
ensureDir(INSTINCTS_DIR);
fs.writeFileSync(filepath, result.trim() + '\n');
log(`Extracted instinct: ${filename}`);
return filename;
### C. Research Topics (mentioned but not explored)
Check for topics the user mentioned but didn't dive into:
- Questions deferred ("look into this later")
- Tools/tech mentioned but not explored
{"type": "research", "topic": "Topic name", "context": "Why this is interesting", "goal_path": "goal path or unrooted"}
### Rules
- One JSON per line, must include "type" field
- No instinct if Write Gate fails, no memory if nothing worth saving, no research if nothing deferred
- Output ONLY JSON lines or empty lines, no other text`;
try {
const env = { ...process.env };
delete env.CLAUDECODE; // Prevent recursive session
const model = process.env.HOMUNCULUS_HARVEST_MODEL || 'claude-sonnet-4-6';
const result = execSync(
`claude --print --model ${model} --max-turns 1 --no-session-persistence`,
{
input: prompt,
encoding: 'utf8',
timeout: 120000,
env,
stdio: ['pipe', 'pipe', 'pipe']
}
).trim();
return result;
} catch (err) {
log(`Extraction failed: ${err.message}`);
log(`Extraction failed: ${err.message?.slice(0, 100)}`);
return null;

@@ -182,8 +276,98 @@ }

// Process extraction result — parse JSON lines and route to appropriate handlers
function processResult(result, { existing, todayCount, dynamicLimit }) {
if (!result) return { instincts: 0, memory: 0, research: 0 };
const stats = { instincts: 0, memory: 0, research: 0 };
for (const line of result.split('\n')) {
const trimmed = line.trim();
if (!trimmed || !trimmed.startsWith('{')) continue;
let entry;
try { entry = JSON.parse(trimmed); } catch { continue; }
switch (entry.type) {
case 'instinct': {
if (todayCount + stats.instincts >= dynamicLimit) break;
if (!entry.name || !entry.content || (entry.confidence || 0) < 0.7) break;
const name = entry.name.replace(/[^a-z0-9-]/g, '-');
if (existing.includes(name)) {
log(` Skip duplicate: ${name}`);
break;
}
// Handle supersedes
const supersedes = Array.isArray(entry.supersedes) ? entry.supersedes : [];
for (const oldName of supersedes) {
if (archiveSupersededInstinct(oldName, name)) {
log(` Superseded: ${oldName} → archived (replaced by ${name})`);
}
}
// Write instinct file
const mechanism = entry.suggested_mechanism || 'skill';
const goalPath = entry.goal_path || 'unrooted';
const supersedesNote = supersedes.length > 0 ? `\n**Supersedes:** ${supersedes.join(', ')}` : '';
const content = `# Instinct: ${entry.title || name}
**Confidence:** ${entry.confidence}
**Source:** auto-learn (evaluate-session)
**Extracted:** ${new Date().toISOString()}
**Suggested mechanism:** ${mechanism}
**Goal path:** ${goalPath}${supersedesNote}
${entry.content}
`;
ensureDir(INSTINCTS_DIR);
fs.writeFileSync(path.join(INSTINCTS_DIR, `${name}.md`), content);
stats.instincts++;
log(` Extracted instinct: ${name} (confidence: ${entry.confidence})`);
break;
}
case 'memory': {
if (entry.title && entry.content && entry.category) {
saveMemorySuggestion(entry);
stats.memory++;
log(` Memory suggestion: [${entry.category}] ${entry.title}`);
}
break;
}
case 'research': {
if (entry.topic) {
saveResearchTopic(entry);
stats.research++;
log(` Research topic: ${entry.topic}`);
}
break;
}
}
}
return stats;
}
// Main
function main() {
// Check daily limit
const todayCount = getTodayExtractions();
if (todayCount >= CONFIG.daily_limit) {
log(`Daily limit reached (${todayCount}/${CONFIG.daily_limit}), skipping`);
// Idempotency: check scan state to avoid reprocessing
try {
if (fs.existsSync(SCAN_STATE_FILE)) {
const state = JSON.parse(fs.readFileSync(SCAN_STATE_FILE, 'utf8'));
const obsModified = fs.existsSync(OBS_FILE) ? fs.statSync(OBS_FILE).mtimeMs : 0;
if (state.last_obs_mtime && obsModified <= state.last_obs_mtime) {
log('Observations unchanged since last evaluation, skipping');
return;
}
}
} catch {}
// Daily cooldown check
const cooldown = getCooldown();
const today = getDateString();
const todayCount = cooldown.date === today ? cooldown.count : 0;
if (todayCount >= CONFIG.hard_cap) {
log(`Hard cap reached (${todayCount}/${CONFIG.hard_cap}), skipping`);
return;

@@ -195,8 +379,35 @@ }

const result = extractInstinct(analysis);
if (result) {
log(`Successfully extracted: ${result}`);
// Apply dynamic daily limit
const dynamicLimit = getDailyLimit(analysis.user_msg_count);
if (todayCount >= dynamicLimit) {
log(`Dynamic limit reached (${todayCount}/${dynamicLimit}), skipping`);
return;
}
const existing = getExistingInstincts();
log(`Analyzing: ${analysis.total_observations} observations, ${analysis.frequent_tools.length} patterns (limit: ${dynamicLimit})`);
const result = extractFromSession(analysis);
const stats = processResult(result, { existing, todayCount, dynamicLimit });
if (stats.instincts > 0) {
updateCooldown(todayCount + stats.instincts);
}
// Update scan state
try {
const obsModified = fs.existsSync(OBS_FILE) ? fs.statSync(OBS_FILE).mtimeMs : 0;
const tmp = SCAN_STATE_FILE + '.tmp';
fs.writeFileSync(tmp, JSON.stringify({
last_scan_time: new Date().toISOString(),
last_obs_mtime: obsModified,
last_stats: stats
}, null, 2) + '\n');
fs.renameSync(tmp, SCAN_STATE_FILE);
} catch {}
log(`Done: ${stats.instincts} instincts, ${stats.memory} memory suggestions, ${stats.research} research topics`);
}
main();

@@ -5,2 +5,4 @@ #!/usr/bin/env bash

# Called by Claude Code PostToolUse hook
#
# v0.9.0: Added noise filtering, reference tracking, skill name enrichment

@@ -12,4 +14,4 @@ set -euo pipefail

OBS_FILE="${HOMUNCULUS_DIR}/observations.jsonl"
REF_FILE="${HOMUNCULUS_DIR}/reference-tracking.jsonl"
MAX_SIZE=$((10 * 1024 * 1024)) # 10MB
COUNTER_FILE="/tmp/homunculus-tool-count-$$"

@@ -31,24 +33,62 @@ PHASE="${1:-unknown}"

# Only observe post-tool usage
TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
# ── Noise filter ──
# Skip low-value observations to keep observations.jsonl meaningful.
# Read-only tools provide no actionable patterns for evolution.
# Exception: Read calls targeting instinct/skill files are tracked for reference analysis.
case "$TOOL_NAME" in
Read)
# Track instinct/skill reads for reference analysis (used by prune-instincts.js)
if command -v jq &>/dev/null && [ -n "$INPUT" ]; then
FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty' 2>/dev/null)
case "$FILE_PATH" in
*/instincts/personal/*|*/instincts/archived/*|*/evolved/skills/*|*/evolved/agents/*)
mkdir -p "$(dirname "$REF_FILE")"
echo "$INPUT" | jq -c --arg ts "$TIMESTAMP" --arg path "$FILE_PATH" \
'{timestamp: $ts, type: "reference", path: $path}' \
>> "$REF_FILE" 2>/dev/null || true
;;
esac
fi
exit 0
;;
# Skip entirely: read-only tools provide no actionable patterns
Glob|Grep|TodoWrite|TodoRead)
exit 0
;;
Skill)
# Enrich tool name with specific skill for evolution stats
if command -v jq &>/dev/null && [ -n "$INPUT" ]; then
SKILL_NAME=$(echo "$INPUT" | jq -r '.tool_input.skill // empty' 2>/dev/null)
[ -n "$SKILL_NAME" ] && TOOL_NAME="Skill:${SKILL_NAME}"
fi
;;
# Skip pre-phase: we only care about results (post) for write operations
Bash|Write|Edit|NotebookEdit)
[ "$PHASE" = "pre" ] && exit 0
;;
esac
# Only observe post-tool usage (after noise filter, so Skill enrichment runs on post)
[ "$PHASE" != "post" ] && exit 0
# Rotate if too large
if [ -f "$OBS_FILE" ] && [ "$(stat -f%z "$OBS_FILE" 2>/dev/null || stat -c%s "$OBS_FILE" 2>/dev/null || echo 0)" -gt "$MAX_SIZE" ]; then
TIMESTAMP=$(date +%Y%m%d%H%M%S)
gzip -c "$OBS_FILE" > "${OBS_FILE}.${TIMESTAMP}.gz"
: > "$OBS_FILE"
fi
# Write observation
TIMESTAMP=$(date -u +%Y-%m-%dT%H:%M:%SZ)
# ── Write observation ──
mkdir -p "$(dirname "$OBS_FILE")"
if command -v jq &>/dev/null; then
jq -nc \
--arg ts "$TIMESTAMP" \
--arg phase "$PHASE" \
--arg tool "$TOOL_NAME" \
'{timestamp: $ts, phase: $phase, tool: $tool}' >> "$OBS_FILE"
if command -v jq &>/dev/null && [ -n "$INPUT" ]; then
# Include agent context for pattern analysis
echo "$INPUT" | jq -c --arg ts "$TIMESTAMP" --arg phase "$PHASE" --arg tool "$TOOL_NAME" \
'{timestamp: $ts, phase: $phase, tool: $tool,
agent_id: (.agent_id // null), agent_type: (.agent_type // null)}' \
>> "$OBS_FILE" 2>/dev/null || true
else
echo "{\"timestamp\":\"$TIMESTAMP\",\"phase\":\"$PHASE\",\"tool\":\"$TOOL_NAME\"}" >> "$OBS_FILE"
fi
# ── Auto-archive if too large ──
if [ -f "$OBS_FILE" ] && [ "$(stat -f%z "$OBS_FILE" 2>/dev/null || stat -c%s "$OBS_FILE" 2>/dev/null || echo 0)" -gt "$MAX_SIZE" ]; then
ARCHIVE="$OBS_FILE.$(date +%Y%m%d%H%M%S).gz"
gzip -c "$OBS_FILE" > "$ARCHIVE"
: > "$OBS_FILE"
fi
#!/usr/bin/env node
// prune-instincts.js — Auto-archive low-value instincts
//
// v0.9.0: Reference frequency scoring, 3-tier skill coverage, grace period, --json
//
// Scoring dimensions:
// 0. Capacity pressure (soft limit)
// 1. Age (older = more likely outdated)
// 2. Confidence with time decay (half-life based)
// 3. Skill coverage (already covered by a skill = lower value)
// 2. Reference frequency (actually used = higher value)
// 3. Skill coverage — 3 tiers: direct evolved / high overlap / partial overlap
// 4. Content size (tiny instincts = less valuable)
// 5. Confidence with time decay (half-life based, 14-day grace period)
//

@@ -13,2 +18,3 @@ // Usage:

// node prune-instincts.js --threshold 40 # Custom score threshold (default: 75)
// node prune-instincts.js --json # Output JSON for programmatic use

@@ -20,11 +26,14 @@ const fs = require('fs');

const BASE_DIR = process.env.HOMUNCULUS_BASE || process.cwd();
const PERSONAL_DIR = path.join(BASE_DIR, 'homunculus', 'instincts', 'personal');
const ARCHIVED_DIR = path.join(BASE_DIR, 'homunculus', 'instincts', 'archived');
const SKILLS_DIR = path.join(BASE_DIR, 'homunculus', 'evolved', 'skills');
const HOMUNCULUS_DIR = path.join(BASE_DIR, 'homunculus');
const PERSONAL_DIR = path.join(HOMUNCULUS_DIR, 'instincts', 'personal');
const ARCHIVED_DIR = path.join(HOMUNCULUS_DIR, 'instincts', 'archived');
const SKILLS_DIR = path.join(HOMUNCULUS_DIR, 'evolved', 'skills');
const REF_FILE = path.join(HOMUNCULUS_DIR, 'reference-tracking.jsonl');
const args = process.argv.slice(2);
const applyMode = args.includes('--apply');
const jsonMode = args.includes('--json');
const thresholdIdx = args.indexOf('--threshold');
const ARCHIVE_THRESHOLD = thresholdIdx >= 0 ? parseInt(args[thresholdIdx + 1], 10) : 75;
const CAPACITY_SOFT_LIMIT = 80;
const CAPACITY_SOFT_LIMIT = parseInt(process.env.HOMUNCULUS_CAPACITY_LIMIT || '80');

@@ -34,2 +43,3 @@ // Confidence decay: half-life in days

const DECAY_LAMBDA = Math.LN2 / CONFIDENCE_HALF_LIFE_DAYS;
const GRACE_PERIOD_DAYS = 14; // No decay for first 2 weeks

@@ -44,3 +54,4 @@ function safeRead(fp) {

const confidence = parseFloat((content.match(/confidence:\s*([\d.]+)/im) || [])[1] || '0.5');
// Support both plain and markdown-bold metadata formats
const confidence = parseFloat((content.match(/\*?\*?confidence\*?\*?:?\s*([\d.]+)/im) || [])[1] || '0.5');
const createdMatch = content.match(/(?:extracted|created|date):\s*"?([^"\n]+)"?/im);

@@ -50,63 +61,146 @@ const created = createdMatch ? new Date(createdMatch[1]) : null;

const updated = updatedMatch ? new Date(updatedMatch[1]) : created;
const source = (content.match(/\*?\*?source\*?\*?:?\s*"?([^"\n]+)"?/im) || [])[1]?.trim() || 'unknown';
// Confidence decay
const daysSinceUpdate = updated ? (Date.now() - updated.getTime()) / 86400000 : 180;
const effectiveConfidence = confidence * Math.exp(-DECAY_LAMBDA * daysSinceUpdate);
return { name, filepath, content, confidence, created, updated, source };
}
return { name, confidence, effectiveConfidence, created, updated, daysSinceUpdate, content };
// Load reference tracking data (from observe.sh)
function loadReferenceData() {
const refs = {};
const content = safeRead(REF_FILE);
if (!content) return refs;
for (const line of content.split('\n').filter(l => l.trim())) {
try {
const entry = JSON.parse(line);
const p = entry.path || '';
if (!refs[p]) refs[p] = { count: 0, lastRef: null };
refs[p].count++;
const ts = entry.timestamp ? new Date(entry.timestamp).getTime() : 0;
if (!refs[p].lastRef || ts > refs[p].lastRef) refs[p].lastRef = ts;
} catch {}
}
return refs;
}
function getSkillCoverage() {
const coverage = new Set();
if (!fs.existsSync(SKILLS_DIR)) return coverage;
for (const file of fs.readdirSync(SKILLS_DIR).filter(f => f.endsWith('.md'))) {
const content = safeRead(path.join(SKILLS_DIR, file));
// Skills often reference instinct names they were derived from
const sourceMatch = content.match(/source_instincts?:(.+)/gim);
if (sourceMatch) {
for (const match of sourceMatch) {
const names = match.replace(/source_instincts?:/i, '').split(/[,\s]+/);
names.forEach(n => coverage.add(n.trim()));
// Build token sets + evolved-from lists for each skill (3-tier coverage detection)
function loadSkillData() {
const skills = {};
if (!fs.existsSync(SKILLS_DIR)) return skills;
try {
const entries = fs.readdirSync(SKILLS_DIR, { withFileTypes: true });
for (const entry of entries) {
let content;
if (entry.isFile() && entry.name.endsWith('.md')) {
content = safeRead(path.join(SKILLS_DIR, entry.name)).toLowerCase();
} else if (entry.isDirectory()) {
const indexPath = path.join(SKILLS_DIR, entry.name, 'index.md');
if (fs.existsSync(indexPath)) content = safeRead(indexPath).toLowerCase();
}
if (!content) continue;
const skillName = entry.name.replace('.md', '');
const tokens = new Set(content.split(/[\s\-_,./:()\[\]{}#*>|]+/).filter(w => w.length > 4));
// Extract "evolved from" instinct names
const evolvedFrom = new Set();
const matches = content.matchAll(/`([^`]+\.md)`/g);
for (const m of matches) evolvedFrom.add(m[1].replace('.md', ''));
skills[skillName] = { tokens, evolvedFrom };
}
}
return coverage;
} catch {}
return skills;
}
function scoreInstinct(instinct, skillCoverage, totalCount) {
function scoreInstinct(instinct, refs, skillData, now, totalRefs, activeCount) {
let score = 100;
const reasons = [];
// Effective confidence (with decay)
if (instinct.effectiveConfidence < 0.5) {
score -= 30;
reasons.push(`low effective confidence: ${instinct.effectiveConfidence.toFixed(2)}`);
} else if (instinct.effectiveConfidence < 0.7) {
// 0. Capacity pressure
if (activeCount > CAPACITY_SOFT_LIMIT) {
const pressure = Math.min((activeCount - CAPACITY_SOFT_LIMIT) * 2, 20);
score -= pressure;
reasons.push(`capacity ${activeCount}/${CAPACITY_SOFT_LIMIT} (-${pressure})`);
}
// 1. Age penalty
if (instinct.created) {
const ageDays = (now - instinct.created.getTime()) / 86400000;
if (ageDays > 30) {
const penalty = Math.min(Math.floor((ageDays - 30) / 7) * 5, 30);
if (penalty > 0) {
score -= penalty;
reasons.push(`age ${Math.round(ageDays)}d (-${penalty})`);
}
}
}
// 2. Reference frequency bonus/penalty
const refData = refs[instinct.filepath];
if (refData && refData.count > 0) {
const bonus = Math.min(refData.count * 5, 25);
score += bonus;
reasons.push(`refs ${refData.count} (+${bonus})`);
} else if (totalRefs > 50) {
// Only penalize if we have enough data to be confident
score -= 15;
reasons.push(`medium confidence: ${instinct.effectiveConfidence.toFixed(2)}`);
reasons.push('no refs (-15)');
}
// Age penalty
if (instinct.daysSinceUpdate > 60) {
score -= 20;
reasons.push(`old: ${Math.floor(instinct.daysSinceUpdate)} days`);
} else if (instinct.daysSinceUpdate > 30) {
score -= 10;
reasons.push(`aging: ${Math.floor(instinct.daysSinceUpdate)} days`);
// 3. Skill coverage — 3 tiers
const instinctContent = (instinct.name + ' ' + instinct.content).toLowerCase();
const instTokens = new Set(instinctContent.split(/[\s\-_,./:()\[\]{}#*>|]+/).filter(w => w.length > 4));
let bestSkill = null, bestRatio = 0, directlyEvolved = false;
for (const [skillName, { tokens: skillTokens, evolvedFrom }] of Object.entries(skillData)) {
if (evolvedFrom.has(instinct.name)) {
directlyEvolved = true;
bestSkill = skillName;
bestRatio = 1.0;
break;
}
let overlap = 0;
for (const t of instTokens) { if (skillTokens.has(t)) overlap++; }
const ratio = instTokens.size > 0 ? overlap / instTokens.size : 0;
if (ratio > bestRatio) { bestRatio = ratio; bestSkill = skillName; }
}
// Skill coverage
if (skillCoverage.has(instinct.name)) {
score -= 25;
reasons.push('covered by skill');
if (directlyEvolved) {
score -= 35;
reasons.push(`directly evolved into ${bestSkill} (-35)`);
} else if (bestRatio >= 0.45) {
const penalty = Math.min(Math.round(bestRatio * 50), 35);
score -= penalty;
reasons.push(`${Math.round(bestRatio * 100)}% covered by ${bestSkill} (-${penalty})`);
} else if (bestRatio >= 0.30) {
const penalty = Math.min(Math.round(bestRatio * 25), 15);
score -= penalty;
reasons.push(`${Math.round(bestRatio * 100)}% partial overlap with ${bestSkill} (-${penalty})`);
}
// Capacity pressure
if (totalCount > CAPACITY_SOFT_LIMIT) {
score -= Math.min(10, totalCount - CAPACITY_SOFT_LIMIT);
reasons.push(`capacity pressure: ${totalCount}/${CAPACITY_SOFT_LIMIT}`);
// 4. Content size penalty
if (instinct.content.length < 300) {
score -= 10;
reasons.push(`tiny ${instinct.content.length}c (-10)`);
}
return { score: Math.max(0, score), reasons };
// 5. Confidence decay (with grace period)
let effectiveConfidence = instinct.confidence;
if (instinct.updated) {
const daysSinceUpdate = (now - instinct.updated.getTime()) / 86400000;
if (daysSinceUpdate > GRACE_PERIOD_DAYS) {
const decayFactor = Math.exp(-DECAY_LAMBDA * (daysSinceUpdate - GRACE_PERIOD_DAYS));
effectiveConfidence = instinct.confidence * decayFactor;
}
}
if (effectiveConfidence < 0.3) {
score -= 20;
reasons.push(`confidence decayed ${instinct.confidence.toFixed(2)}→${effectiveConfidence.toFixed(2)} (-20)`);
} else if (effectiveConfidence < 0.5) {
score -= 15;
reasons.push(`low effective confidence ${effectiveConfidence.toFixed(2)} (-15)`);
} else if (effectiveConfidence < 0.7) {
score -= 5;
reasons.push(`mid effective confidence ${effectiveConfidence.toFixed(2)} (-5)`);
}
return { score: Math.max(0, Math.min(100, score)), reasons, effectiveConfidence };
}

@@ -120,49 +214,96 @@

const now = Date.now();
const files = fs.readdirSync(PERSONAL_DIR).filter(f => f.endsWith('.md'));
const skillCoverage = getSkillCoverage();
const candidates = [];
const activeCount = files.length;
for (const file of files) {
const instinct = parseInstinct(path.join(PERSONAL_DIR, file));
const { score, reasons } = scoreInstinct(instinct, skillCoverage, files.length);
console.log(`\n=== Instinct Pruning Analysis ===`);
console.log(`Active: ${activeCount} | Soft limit: ${CAPACITY_SOFT_LIMIT} | Threshold: ${ARCHIVE_THRESHOLD}`);
console.log(`Confidence decay: half-life ${CONFIDENCE_HALF_LIFE_DAYS}d, grace ${GRACE_PERIOD_DAYS}d`);
console.log(`Mode: ${applyMode ? 'APPLY (will archive)' : 'DRY RUN (preview only)'}\n`);
if (score < ARCHIVE_THRESHOLD) {
candidates.push({ ...instinct, score, reasons });
}
const refs = loadReferenceData();
const totalRefs = Object.values(refs).reduce((a, b) => a + b.count, 0);
console.log(`Reference tracking: ${totalRefs} total reads`);
if (totalRefs < 50) {
console.log(` (Low data — reference scoring will be conservative)\n`);
} else {
console.log();
}
candidates.sort((a, b) => a.score - b.score);
const skillData = loadSkillData();
console.log(`Skills loaded: ${Object.keys(skillData).length}\n`);
console.log(`\nInstincts: ${files.length} active | Threshold: ${ARCHIVE_THRESHOLD} | Candidates: ${candidates.length}`);
console.log('─'.repeat(60));
// Score all instincts
const scored = files.map(f => {
const instinct = parseInstinct(path.join(PERSONAL_DIR, f));
const { score, reasons, effectiveConfidence } = scoreInstinct(instinct, refs, skillData, now, totalRefs, activeCount);
return { ...instinct, score, reasons, effectiveConfidence };
}).sort((a, b) => a.score - b.score);
if (candidates.length === 0) {
console.log('No archive candidates found.');
return;
const candidates = scored.filter(i => i.score < ARCHIVE_THRESHOLD);
const atRisk = scored.filter(i => i.score >= ARCHIVE_THRESHOLD && i.score < 50);
// Display candidates
if (candidates.length > 0) {
console.log(`--- Archive Candidates (score < ${ARCHIVE_THRESHOLD}) ---\n`);
for (const c of candidates) {
console.log(` [${c.score.toString().padStart(3)}] ${c.name}`);
console.log(` ${c.reasons.join(' | ')}`);
}
console.log();
} else {
console.log(`No archive candidates (all scores >= ${ARCHIVE_THRESHOLD})\n`);
}
for (const c of candidates) {
console.log(` ${c.score.toString().padStart(3)} | ${c.name}`);
console.log(` ${c.reasons.join(', ')}`);
// Display at-risk
if (atRisk.length > 0) {
console.log(`--- At Risk (${ARCHIVE_THRESHOLD}–49) ---\n`);
for (const c of atRisk) {
console.log(` [${c.score.toString().padStart(3)}] ${c.name}`);
console.log(` ${c.reasons.join(' | ')}`);
}
console.log();
}
if (applyMode) {
console.log('\nArchiving...');
// Summary
const afterArchive = activeCount - candidates.length;
console.log(`--- Summary ---`);
console.log(`Would archive: ${candidates.length} instincts`);
console.log(`After pruning: ${afterArchive} active (${afterArchive > CAPACITY_SOFT_LIMIT ? 'above soft limit' : 'within capacity'})`);
// Apply
if (applyMode && candidates.length > 0) {
console.log(`\nArchiving ${candidates.length} instincts...\n`);
if (!fs.existsSync(ARCHIVED_DIR)) fs.mkdirSync(ARCHIVED_DIR, { recursive: true });
for (const c of candidates) {
const src = path.join(PERSONAL_DIR, `${c.name}.md`);
const dest = path.join(ARCHIVED_DIR, `${c.name}.md`);
// Append archive metadata before moving
const note = `\n\n---\n_Archived: ${new Date().toISOString().slice(0, 10)} | Covered-by: ${c.reason || 'low-score'} | Score: ${c.score}_\n`;
fs.appendFileSync(src, note);
fs.renameSync(src, dest);
console.log(` Archived: ${c.name}`);
const src = c.filepath;
const dst = path.join(ARCHIVED_DIR, path.basename(src));
const coveredBy = c.reasons
.filter(r => r.includes('evolved into') || r.includes('covered by') || r.includes('overlap with'))
.map(r => r.replace(/\s*\(-\d+\)/, ''))
.join('; ') || 'low score';
const note = `\n\n---\n_Archived: ${new Date().toISOString().slice(0, 10)} | Score: ${c.score} | Covered-by: ${coveredBy}_\n`;
const content = safeRead(src) + note;
fs.writeFileSync(dst, content);
fs.unlinkSync(src);
console.log(` Archived: ${c.name} (score: ${c.score})`);
}
console.log(`\nDone. Archived ${candidates.length} instincts.`);
} else {
console.log(`\nDone. ${candidates.length} instincts archived.`);
} else if (!applyMode && candidates.length > 0) {
console.log(`\nDry run. Use --apply to archive these ${candidates.length} instincts.`);
}
// JSON output
if (jsonMode) {
const output = {
active: activeCount,
candidates: candidates.map(c => ({ name: c.name, score: c.score, reasons: c.reasons })),
at_risk: atRisk.map(c => ({ name: c.name, score: c.score, reasons: c.reasons })),
after_pruning: afterArchive
};
console.log('\n' + JSON.stringify(output, null, 2));
}
}
main();

@@ -38,3 +38,3 @@ # Nightly Agent

| **P0** | Complete assigned high-priority tasks |
| **P1** | Evolution cycle — route instincts to 8 mechanisms (hook/rule/skill/script/agent/...), eval + improve skills, review workflow/subagent health, check all mechanisms working, review goal implementations |
| **P1** | Evolution cycle — extract instincts + memory suggestions + research topics, route instincts to 8 mechanisms (hook/rule/skill/script/agent/...), eval + improve skills, review goal implementations |
| **P2** | Research — scan tech news, changelogs, community (with cross-night dedup) |

@@ -117,2 +117,13 @@ | **P3** | Experiments — generate hypotheses from weak goals, test in isolated worktrees |

## Reports Directory
The evolution engine writes suggestions to `homunculus/reports/` (non-invasive — you decide what to adopt):
| File | Purpose |
|------|---------|
| `memory-suggestions.jsonl` | Context worth preserving (user preferences, project decisions, external resources) |
| `research-queue.jsonl` | Topics mentioned but not explored — candidates for nightly research |
Review these during `/hm-night` or check them manually anytime.
## Morning Report

@@ -123,2 +134,3 @@

- System evolution (instinct routing, skill evals)
- Memory suggestions extracted from sessions
- Research topics (with source URLs and goal relevance tags)

@@ -125,0 +137,0 @@ - Experiments (hypothesis, result, merged or discarded)

{
"name": "homunculus-code",
"version": "0.8.0",
"version": "0.9.0",
"description": "A self-evolving AI assistant that grows smarter every night",

@@ -5,0 +5,0 @@ "bin": {

@@ -167,2 +167,13 @@ # Homunculus for Claude Code

### v0.9.0 — Evolution Engine Upgrade (Mar 2026)
- **Smart observation** — `observe.sh` now filters noise (skips Read/Glob/Grep, only records post-phase for writes) and tracks reference frequency — which instincts and skills are actually being read
- **Three-layer extraction** — `evaluate-session.js` now extracts instincts + memory suggestions + research topics in a single pass. Memory and research are written to `homunculus/reports/` for you to review (non-invasive)
- **Semantic dedup** — New instincts can declare `supersedes` to auto-archive older instincts they replace
- **Write Gate** — Extraction prompt now includes quality criteria (must change future behavior, capture a commitment, or preserve a decision rationale)
- **Dynamic daily cap** — Instinct extraction limit scales with session size (1 for short sessions, up to 5 for long ones)
- **Smarter pruning** — `prune-instincts.js` now uses reference frequency scoring (+25 for frequently used, -15 for never referenced), 3-tier skill coverage detection (direct evolved / high overlap / partial), 14-day grace period before confidence decay, and at-risk warnings
- **Idempotency** — Extraction tracks scan state to avoid reprocessing the same observations
- **`--json` output** — `prune-instincts.js` supports `--json` for programmatic use
### v0.8.0 — Upgrade Mechanism (Mar 2026)

@@ -169,0 +180,0 @@

@@ -14,11 +14,13 @@ # Evolution System (Homunculus)

```
Observe (observations.jsonl) → Pattern detection (confidence > 0.7) → instincts/personal/*.md
/evolve → evolved/skills/*.md
eval → improve loop → 100% pass
Observe (observations.jsonl) → Three-layer extraction:
├── Instincts (behavioral patterns, confidence > 0.7, Write Gate)
│ ↓ supersedes check → auto-archive replaced instincts
│ ↓ /evolve → route to best mechanism (hook/rule/skill/script/agent/...)
│ ↓ if skill → eval → improve loop → 100% pass
├── Memory suggestions → reports/memory-suggestions.jsonl (user reviews)
└── Research topics → reports/research-queue.jsonl (nightly agent picks up)
```
## Manual Triggers
- `/evolve` — Aggregate instincts into a skill
- `/evolve` — Route instincts to the best mechanism (skill is just one of 8)
- `/eval-skill` — Test a skill against its eval spec

@@ -28,4 +30,5 @@ - `/improve-skill` — Auto-improve until eval passes

## Automatic Maintenance
- Instinct extraction from session observations
- Instinct pruning (confidence decay, skill coverage check)
- Three-layer extraction: instincts + memory + research from sessions
- Semantic dedup: new instincts auto-archive superseded ones
- Smart pruning: reference frequency + 3-tier skill coverage + confidence decay (14-day grace)
- Skill eval → improve pipeline