compile

Compile — Knowledge Compiler

Reads raw .agents/ artifacts and compiles them into a structured, interlinked markdown wiki. Inspired by Karpathy's LLM Knowledge Bases.

What This Skill Does

The knowledge flywheel captures signal reactively (via /retro, /post-mortem, /forge). /compile closes the loop by:

1. Mining unextracted signal from git and .agents/ (existing)
2. Growing learnings via validation, synthesis, and gap detection (existing)
3. Compiling raw artifacts into interlinked wiki articles (NEW — the core value)
4. Linting the compiled wiki for contradictions, orphans, and gaps (NEW)
5. Defragging stale and duplicate artifacts (existing)

No vector DB. At personal scale (~100-400 articles), the compiled wiki fits in context windows. The wiki IS the retrieval layer.

Output: .agents/compiled/ — encyclopedia-style markdown with [[backlinks]], index.md catalog, and log.md chronological record.

Pluggable Compute Backend

Set AGENTOPS_COMPILE_RUNTIME to choose the LLM backend:

| Value | Backend | Notes | |-------|---------|-------| | claude-cli | Local claude binary | Zero-config. Inherits your Claude Code auth — no API key needed. Auto-selected if claude is on PATH and nothing else is set. | | ollama | Ollama API | Default model: gemma3:27b. Set OLLAMA_HOST for remote (e.g., ssh -L 11435:localhost:11435 bushido-windows). | | claude | Claude API (HTTP) | Uses ANTHROPIC_API_KEY. Model: claude-sonnet-4-20250514. | | openai | OpenAI-compatible | Uses OPENAI_API_KEY + OPENAI_BASE_URL. | | (unset) | Claude Code session | Compilation happens inline via the current session's LLM. |

When AGENTOPS_COMPILE_RUNTIME is unset, ao compile first tries to auto-detect a local claude binary (claude-cli runtime). If that is also absent, headless compile fails fast with an explicit error naming the env var to set. Interactive /compile invocations still run compilation prompts inline — the agent reading this SKILL.md IS the compiler.

Runtime preference (override auto-detect)

To force a non-auto-detected runtime permanently (e.g. you have claude installed but prefer Ollama for privacy), set it in ~/.agentops/config.yaml:

compile:
  preferred_runtime: ollama

Precedence (high → low): --runtime flag, AGENTOPS_COMPILE_RUNTIME env, compile.preferred_runtime config, claude-binary auto-detect, empty (error).

Large-corpus batching

ao compile passes --batch-size to the headless compiler (default 25 changed files per LLM prompt). A fresh run against a 2000+ file corpus will split into batches automatically instead of sending one giant prompt.

Flags:

• --batch-size N — files per batch (default 25)
• --max-batches N — cap batches per invocation; remaining files are picked

up on the next run (default 0 = unlimited)

Execution Steps

Phase-by-phase detail lives in [references/phases.md](references/phases.md). Summary of modes:

• /compile — Full cycle: Mine → Grow → Compile → Lint → Defrag
• /compile --compile-only — Skip mine/grow, just compile + lint
• /compile --lint-only — Only lint the existing compiled wiki
• /compile --defrag-only — Only run defrag/cleanup
• /compile --mine-only — Only run mine + grow (legacy behavior)

The steps are:

1. Mine — extract signal from git + .agents/research/ + complexity hotspots
2. Grow — LLM-driven validation, synthesis, gap detection; adjust learning confidence
3. Compile — inventory → topic extraction → wiki articles with [[backlinks]]
4. Lint — contradictions, orphans, missing cross-refs, stale claims
5. Defrag — prune stale, dedup near-duplicates, sweep oscillating goals, normalization scan
6. Report — write .agents/compile/YYYY-MM-DD-report.md

See [references/phases.md](references/phases.md) for the full per-phase procedure, confidence-scoring table, auto-promotion rules, template shapes for article / index / log / lint-report / compile-report, and the normalization defect scan.

Scheduling / Auto-Trigger

Lightweight defrag (prune + dedup, no mining or compilation) runs automatically at session end via the compile-session-defrag.sh hook. This keeps the knowledge store clean without requiring manual /compile invocations. The hook:

• Fires on every SessionEnd event after session-end-maintenance.sh
• Skips silently if the ao CLI is not available
• Runs only ao defrag --prune --dedup (no compilation or mining)
• Has a 20-second timeout to avoid blocking session teardown

For full compilation, invoke /compile manually or schedule the headless compiler script with your host OS:

# Example: external cron entry for nightly compilation on bushido
0 3 * * * cd /path/to/repo && AGENTOPS_COMPILE_RUNTIME=ollama bash skills/compile/scripts/compile.sh --force

AgentOps exposes this flow through ao compile. If you want unattended compilation, use your host scheduler (launchd, cron, systemd, CI, etc.) to invoke ao compile --force --runtime ollama or call the lower-level bash skills/compile/scripts/compile.sh directly. If you want the broader private overnight loop, use ao overnight start instead of inventing a parallel Dream wrapper inside /compile.

Interactive Modes

These modes describe the interactive /compile skill behavior:

| Mode | Description | |------|-------------| | --compile-only | Skip mine/grow, just compile + lint | | --lint-only | Only lint the existing compiled wiki | | --defrag-only | Only run defrag/cleanup | | --mine-only | Only run mine + grow (legacy behavior) | | --full | Full cycle: mine → grow → compile → lint → defrag | | --since 26h | Time window for the mine phase | | --incremental | Skip unchanged source files (hash-based) | | --force | Recompile all articles regardless of hashes |

Headless Script Flags

For unattended runs, bash skills/compile/scripts/compile.sh supports:

| Flag | Default | Description | |------|---------|-------------| | --sources <dir> | .agents | Source root for learnings, patterns, research, retros, forge, and knowledge | | --output <dir> | .agents/compiled | Target directory for compiled wiki output | | --incremental | on | Skip unchanged source files (hash-based) | | --force | off | Recompile all articles regardless of hashes | | --lint-only | off | Only run the lint pass on the existing compiled wiki | | --full | on | Accepted for parity; default behavior already runs the full headless compile path |

Examples

User says: /compile — Full Mine → Grow → Compile → Lint → Defrag cycle.

User says: /compile --compile-only — Just compile raw artifacts into wiki.

User says: /compile --lint-only — Scan existing wiki for health issues.

User says: /compile --since 7d — Mines with a wider window (7 days).

Scheduled externally: Nightly compilation on bushido GPU via Ollama.

Pre-evolve warmup: Run /compile before /evolve for a fresh, validated knowledge base.

Troubleshooting

| Problem | Cause | Solution | |---------|-------|----------| | ao mine not found | ao CLI not in PATH | Use manual fallback in Step 1 | | No orphaned research | All research already referenced | Skip 2b, proceed to synthesis | | Empty mine output | No recent activity | Widen --since window | | Oscillation sweep empty | No oscillating goals | Healthy state — no action needed | | Ollama connection refused | Tunnel not running or wrong host | Run ssh -L 11435:localhost:11435 bushido-windows or check OLLAMA_HOST | | Compilation too slow | Large corpus on small model | Use --incremental or switch to larger model | | Hash file missing | First compilation | Normal — full compile runs, hashes saved after |

Reference Documents

• [references/compile.feature](references/compile.feature) — Executable spec: Mine→Grow→Lint→Defrag rebuild, lint-not-