Context Manager

Manage the scarcest resource in the LLM OS: context window (RAM).

Usage Template

Prompt

Use context-manager for this task. Estimate token budget, choose what to keep, what to summarize, and what to drop.

Use Case

• Preparing a long task, large document set, or multi-step agent workflow without overrunning context.

Expected Result

• The agent produces a context budget, priority order, truncation plan, and cost-aware prompt assembly.

Output Example

• A table of keep/summarize/drop decisions with estimated tokens and retrieval priority.

Verification Case

• The final context plan names included sources, excluded sources, and the reason for each exclusion.

Verified Effect

• Long or messy context becomes a scoped prompt plan with lower token waste and clearer retrieval priorities.

When to Use

• "Context is full", "too many tokens", "slow responses"
• Before complex multi-step tasks
• Designing prompt templates with variable-length content
• "Optimize my prompts" or "reduce token cost"
• Before ingesting large documents

Core Principles

0. Concrete Ideas Buy Speed (Andrew Ng)

"When you're vague, you're almost always right. When you're concrete, you may be right or wrong. Either way is fine — we can discover that much more fast."

1. Token Budgeting

Context Window = System Prompt + User Input + Retrieved Context + Tool Results + Agent Thoughts

Budget:
├─ System Prompt + Schema     10-15%
├─ User Request                 5-10%
├─ Retrieved Context (RAG)    30-40%
├─ Tool Results                20-30%
└─ Agent Reasoning             10-15%

Rule: Exceeds 80% → must truncate/compress.

2. Prompt Assembly

Layer 1: Immutable Core (always injected)
├─ System prompt (persona + constraints)
├─ Schema/ontology
└─ Safety rules

Layer 2: Task Context (per-request)
├─ User request
├─ Relevant wiki pages (top-k)
└─ Recent session log

Layer 3: Ephemeral (auto-managed)
├─ Tool call history (trim old)
├─ Large outputs (summarize)
└─ Extended thinking (compact)

3. Truncation Strategies

Token Cost (Claude)

Quick estimates:

• 1 token ≈ 0.75 word
• 1 page ≈ 300-500 tokens
• 1 source ≈ 2000-8000 tokens
• 1 hour agentic ≈ 100K-500K tokens
• Weekly active user ≈ $9-15

Decision Tree

Simple task (1-2 steps)?
├─ YES → Haiku, ~10K tokens
└─ NO → Analytical (research/compile)?
    ├─ YES → Opus, ~50-100K
    └─ NO → Procedural (ingest/lint)?
        ├─ YES → Sonnet, ~20-50K
        └─ NO → Sonnet default

Tokenmaxxing vs Efficiency

"Token maxing is actually the coolest thing you can do now." — Gary Tan

Thin Harness, Fat Skills:

• Simple tasks → thin prompt
• Complex tasks → fat prompt with examples

When to Tokenmax

When to Be Efficient

Quality Gates

• Token budget <80% before first call
• 3-layer prompt assembly
• Large outputs (>500 tokens) summarized
• Cost estimated for >100K token tasks
• Cache-friendly ordering
• Context utilization logged