$automation-shape-routing — Orchestration vs NTM vs Skill
The trap this kills: "I built a lot of skills; they should become
orchestration scripts." Mostly false. Most orchestration-looking skills are
either long-lived/human-attachable (stay NTM) or hard-sequential (stay skills).
The win is the routing rule, not a migration project.
The three shapes
| Shape | What it is | Mechanism | |---|---|---| | Orchestration | Deterministic, reproducible fan-out / pipeline / loop over sub-agents, each returning structured output | Codex orchestration — codex exec driving spawn_agents (parallel fan-out), staged pipelines, and loop-until-budget, with an output_schema per sub-agent. Headless, reproducible, bounded concurrency. | | NTM swarm | Long-lived, human-in-the-loop multi-agent run | ntm / *-with-ntm — persistent tmux panes, robot API, mail/locks, attach + nudge + kill/relaunch. | | Plain skill | One model reasoning through a procedure or knowledge | A single SKILL.md. No fan-out, or a strictly sequential edit-loop. |
The decision rule (three axes)
Ask in order:
- Is there real orchestration at all? (fan-out / barrier / multi-stage, OR a
loop with parallelism to exploit) — if no → plain skill. Stop.
- Must a human attach and steer mid-run? Or does it run for hours, do
open-ended file edits, juggle a fluid population (rate limits, kill/ relaunch, prompt-cache rounds), or relay between cross-model panes? — if yes → NTM swarm.
- Otherwise — fixed DAG, sub-agents return structured JSON (not free-form
edits needing review), no attach needed, you want it reproducible + headless → Orchestration.
One-line litmus:
deterministic DAG + structured JSON + no human-attach + headless-wanted → Orchestration
long-lived + attachable + open-ended file edits / fluid population → NTM
no fan-out, or hard-sequential edit loop → plain skill
Spike-validated nuances (2026-05-29)
A live three-legged spike (~/dev/agentops-3cat-spike/) measured the same task on all three backends. Two findings refine the rule:
- The primary axis is control-plane vs in-session, not "parallel vs serial."
NTM is a control-plane that runs Claude/Codex/Gemini as panes — it is not a peer of the native runtimes, it is the supervisor tier above them. Choose NTM when you need the control plane (attach/steer, persistence, multi-vendor); choose in-session native orchestration (codex exec + spawn_agents) when you don't.
- Parallel buys quality/independence, NOT wall-clock — at small N. Measured: a
3-way parallel fan-out tied a single sequential agent on wall-clock (191s vs 180s) and cost ~2.7× the tokens — because the synthesis barrier eats the parallel gain. What it bought was depth + independent fresh-eyes (the sequential leg self-reported "monoculture" bias). So: reach for a parallel fan-out when you want independent verification / fresh eyes, not for speed. For speed, you need large N and no barrier — use a streaming pipeline (no barrier), not a collect-all barrier.
Degradation (NTM → native → beads floor) is governed by the OrchestrationPort selector; opt out entirely with AGENTOPS_ORCHESTRATION=off → beads floor, which always works.
Two traps to avoid
- Don't orchestrate a sequential edit-loop. If each pass must see the prior
pass's edits (progressive-deepening reapply, audit-fix-rescan), there's no concurrency to win — an orchestration wrapper adds a process boundary for nothing. Exception: it graduates to a loop-until-budget orchestration only once each step returns structured output instead of free-form edits, and you want it headless/reproducible.
- Don't NTM-ify a clean fan-out, and don't orchestrate an attach-and-steer run.
Headless orchestration runs in-process and cannot be tmux-attached; NTM is built for exactly the live-steering headless orchestration can't do. Picking wrong fights the tool the whole way.
Worked examples
→ Orchestration (deterministic fan-out / synthesize, structured returns): council (N judges → consensus — near-trivial port), the planning half of rpi, judge/refutation panels, any "fan out N analyses → triangulate" task.
→ Stay NTM (long-lived, attachable, open-ended edits, fluid population): the *-with-ntm family (hypothesis research, cross-model review swarms, browser testing), plus swarm/crank in full epic-execution mode — they touch the working tree and need wave-validity gating + human review.
→ Stay plain skill (no exploitable parallelism, or knowledge/one-shot): deliberately one-at-a-time loops (progressive reapply, multi-pass bug hunting); all reference docs; all single-shot transforms (jargon scrub, README authoring).
Canonical orchestration shape
The operating loop is the worked example — a deterministic orchestration that drives codex exec over spawn_agents with a per-sub-agent output_schema (structured JSON, not free-form edits), parallel fan-out barriers (framing-lenses / judges / refutation / slices), explicit phase markers, budget-scaled fan-out width, and bounded re-plan/retry. Start from this shape when authoring an orchestration. It is also the proof that the AgentOps operating loop has two conformant runtimes (skill-driven via rpi/crank/swarm/council, and orchestration-driven via codex exec + spawn_agents) — the basis of the agentops-core-sdk portability thesis. Hand off to $workflow-builder for the authoring path.
Handoff — after the verdict, invoke the next skill
This skill is the front door. It does not build; it routes. Once the shape is decided, hand off:
| Verdict | Next | What it does | |---|---|---| | plain skill | $skill-builder | Scaffold a new SKILL.md against the unified template → then $skill-auditor → $heal-skill. | | Orchestration | $workflow-builder | Author a deterministic codex exec + spawn_agents orchestration with per-sub-agent output_schema. | | NTM swarm | ntm + vibing-with-ntm | Stand up + tend a persistent, human-attachable tmux swarm. |
State the verdict and the deciding axis in one line, then invoke the chosen builder. Do not scaffold here.
Contract note (SDK)
An orchestration is a composite capability (a composition of sub-capabilities with typed control flow); a skill is a leaf. The portable contract for this — a shape: skill|workflow discriminator, a StepGraph, a control_flow enum, a budget, and an OrchestrationPort interface — is net-new SDK work. Port the shape, not the engine: keep concrete orchestrators (Codex sub-agents, swarm dispatch, scheduler — BC4/BC5) behind adapters.