skill-optimizer

When to Use This Skill

• Use when skills are not triggering as expected or seem broken
• Use when you want to audit and improve your skill library's quality
• Use when you want to understand which skills are underperforming or wasting context tokens

Rules

• Read-only: never modify skill files. Only output report.
• All 8 dimensions: do not skip any. If data is insufficient, report "N/A — insufficient session data" rather than omitting.
• Quantify: "you had 12 research tasks last week but the skill never triggered" beats "you often do research".
• Suggest, don't prescribe: give specific wording suggestions for description improvements, but frame as suggestions.
• Show evidence: for undertrigger claims, quote the actual user message that should have triggered the skill.
• Evidence-based suggestions: when suggesting description rewrites, cite the specific research finding that motivates the change (e.g., "front-load trigger keywords — MCP study shows 3.6x selection rate improvement").

Overview

Analyze skills using historical session data + static quality checks, output a diagnostic report with P0/P1/P2 prioritized fixes. Scores each skill on a 5-point composite scale across 8 dimensions.

CSO (Claude/Agent Search Optimization) = writing skill descriptions so agents select the right skill at the right time. This skill checks for CSO violations.

Usage

• /optimize-skill → scan all skills
• /optimize-skill my-skill → single skill
• /optimize-skill skill-a skill-b → multiple specified skills

Data Sources

Auto-detect the current agent platform and scan the corresponding paths:

Platform detection: Check which directories exist. Scan all available sources — a user may have both Claude Code and Codex installed.

Workflow

Identify target skills
        ↓
Collect session data (python3 scripts scan JSONL transcripts)
        ↓
Run 8 analysis dimensions
        ↓
Compute composite scores
        ↓
Output report with P0/P1/P2

Step 1: Identify Target Skills

Scan skill directories in order: ~/.claude/skills/, ~/.codex/skills/, ~/.agents/skills/. Deduplicate by skill name (same name in multiple locations = same skill). For each, read SKILL.md and extract:

• name, description (from YAML frontmatter)
• trigger keywords (from description field)
• defined workflow steps (Step 1/2/3... or ### sections under Workflow)
• word count

If user specified skill names, filter to only those.

Step 2: Collect Session Data

Use python3 scripts via Bash to scan session JSONL files. Extract:

Claude Code sessions (~/.claude/projects/**/*.jsonl):

• Skill tool_use calls (which skills were invoked)
• User messages (full text)
• Assistant messages after skill invocation (for workflow tracking)
• User messages after skill invocation (for reaction analysis)

Codex sessions (~/.codex/sessions/**/*.jsonl):

• session_meta events → extract base_instructions for skill loading evidence
• response_item events → assistant outputs (workflow tracking)
• event_msg events → tool execution and skill-related events
• User messages from turn_context events (for reaction analysis)

Note: Codex injects skills via context rather than explicit Skill tool calls. Skill loading (present in base_instructions) does NOT equal active invocation. To detect actual use, search for skill-specific workflow markers (step headers, output formats) in response_item content within that session. A skill is "invoked" only if the agent produced output following the skill's defined workflow.

Aggregated:

• Per-skill: invocation count, trigger keyword match count
• Per-skill: user reaction sentiment after invocation
• Per-skill: workflow step completion markers

Step 3: Run 8 Analysis Dimensions

You MUST run ALL 8 dimensions. The baseline behavior without this skill is to skip dimensions 4.2, 4.3, 4.5b, and 4.8. These are the most valuable dimensions — do not skip them.

4.1 Trigger Rate

Count how many times each skill was actually invoked vs how many times its trigger keywords appeared in user messages.

Claude Code: count Skill tool_use calls in transcripts. Codex: count sessions where the agent produced output following the skill's workflow markers (not merely loaded in context).

Diagnose:

• Never triggered → skill may be useless or trigger words wrong
• Keywords match >> actual invocations → undertrigger problem, description needs work
• High frequency → core skill, worth optimizing

4.2 Post-Invocation User Reaction

This dimension is critical and easy to skip. Do not skip it.

After a skill is invoked in a session, read the user's next 3 messages. Classify:

• Negative: "no", "wrong", "never mind", "not what I wanted", user interrupts
• Correction: user re-describes their intent, manually overrides skill output
• Positive: "good", "ok", "continue", "nice", user follows the workflow
• Silent switch: user changes topic entirely (likely false positive trigger)

Report per-skill satisfaction rate.

4.3 Workflow Completion Rate

This dimension is critical and easy to skip. Do not skip it.

For each skill invocation found in session data:

1. Extract the skill's defined steps from SKILL.md
2. Search the assistant messages in that session for step markers (Step N, specific output formats defined in the skill)
3. Calculate: how far did execution get?

Report: {skill-name} (N steps): avg completed Step X/N (Y%)

If a specific step is frequently where execution stops, flag it.

4.4 Static Quality Analysis

Check each SKILL.md against these 14 rules:

4.5a False Positive Rate (Overtrigger)

Skill was invoked but user immediately rejected or ignored it.

4.5b Undertrigger Detection

This is the highest-value dimension. For each skill, extract its capability keywords (not just trigger keywords — what the skill CAN do). Then scan user messages for tasks that match those capabilities but where the skill was NOT invoked.

Report: which user messages SHOULD have triggered the skill but didn't, and suggest description improvements.

Compounding Risk Assessment: For skills with chronic undertriggering (0 triggers across 5+ sessions where relevant tasks appeared), flag as "compounding risk" — undertriggered skills cannot self-improve through usage feedback, causing the gap to widen over time. Recommend immediate description rewrite as P0.

4.6 Cross-Skill Conflicts

Compare all skill pairs:

• Trigger keyword overlap (same keywords in two