research-pipeline

Full Research Pipeline: Idea → Experiments → Submission

End-to-end autonomous research workflow for: $ARGUMENTS

Constants

• AUTO_PROCEED = true — When true, Gate 1 auto-selects the top-ranked idea (highest pilot signal + novelty confirmed) and continues to implementation. When false, always waits for explicit user confirmation before proceeding.
• ARXIV_DOWNLOAD = false — When true, /research-lit downloads the top relevant arXiv PDFs during literature survey. When false (default), only fetches metadata via arXiv API. Passed through to /idea-discovery → /research-lit.
• HUMAN_CHECKPOINT = false — When true, the auto-review loops (Stage 3) pause after each round's review to let you see the score and provide custom modification instructions before fixes are implemented. When false (default), loops run fully autonomously. Passed through to /auto-review-loop.
• REVIEWER_DIFFICULTY = medium — How adversarial the reviewer is. medium (default): standard MCP review. hard: adds Reviewer Memory + Debate Protocol. nightmare: GPT reads repo directly via codex exec + memory + debate. Passed through to /auto-review-loop.
• CODE_REVIEW = true — GPT-5.4 xhigh reviews experiment code before deployment. Catches logic bugs before wasting GPU hours. Set false to skip. Passed through to /experiment-bridge.
• BASE_REPO = false — GitHub repo URL to use as base codebase. When set, /experiment-bridge clones the repo first and implements experiments on top of it. When false (default), writes code from scratch or reuses existing project files. Passed through to /experiment-bridge.
• COMPACT = false — When true, generates compact summary files for short-context models and session recovery. Passed through to /idea-discovery and /experiment-bridge.
• AUTO_WRITE = false — When true, automatically invoke Workflow 3 (/paper-writing) after Stage 4. Requires VENUE to be set. When false (default), Stage 4 generates NARRATIVE_REPORT.md and stops — user invokes /paper-writing manually.
• VENUE = ICLR — Target venue for paper writing (Stage 5). Only used when AUTO_WRITE=true. Options: ICLR, NeurIPS, ICML, CVPR, ACL, AAAI, ACM, IEEE_CONF, IEEE_JOURNAL.
• RENDER_HTML = true — When true (default), auto-render NARRATIVE_REPORT.md to HTML at Stage 4 completion via /render-html. Uses --no-review (this is an internal handoff doc to /paper-writing, not a reviewer-facing final artifact — the upstream Stage 3 auto-review loop already cross-model-reviewed the claims). Set false to skip, or pass — render html: false. Non-blocking: if /render-html fails or Codex MCP is unavailable, log the failure and continue — the HTML view is a nice-to-have, not a Stage 4 prerequisite.

💡 Override via argument, e.g., /research-pipeline "topic" — AUTO_PROCEED: false, human checkpoint: true, difficulty: nightmare, code review: false, base repo: https://github.com/org/project, auto_write: true, venue: NeurIPS.

Overview

This skill chains the entire research lifecycle into a single pipeline:

/idea-discovery → /experiment-bridge → /auto-review-loop → /paper-writing (optional)
├── Workflow 1 ──┤├── Workflow 1.5 ──┤├── Workflow 2 ───┤ ├── Workflow 3 ──┤

It orchestrates up to four major workflows in sequence. Workflow 3 (paper writing) is optional and controlled by AUTO_WRITE.

Pipeline

Stage 1: Idea Discovery (Workflow 1)

If RESEARCH_BRIEF.md exists in the project root, it will be automatically loaded as detailed context (replaces one-line prompt). See templates/RESEARCH_BRIEF_TEMPLATE.md.

Invoke the idea discovery pipeline:

/idea-discovery "$ARGUMENTS"

This internally runs: /research-lit → /idea-creator → /novelty-check → /research-review

Output: idea-stage/IDEA_REPORT.md with ranked, validated, pilot-tested ideas.

Review Tracing follows the downstream review skills. Stage 1 and Stage 3 preserve reviewer prompts/responses through their own trace protocols so the final handoff can be audited.

🚦 Gate 1 — Human Checkpoint:

After idea-stage/IDEA_REPORT.md is generated, pause and present the top ideas to the user:

📋 Idea Discovery complete. Top ideas:

1. [Idea 1 title] — Pilot: POSITIVE (+X%), Novelty: CONFIRMED
2. [Idea 2 title] — Pilot: WEAK POSITIVE (+Y%), Novelty: CONFIRMED
3. [Idea 3 title] — Pilot: NEGATIVE, eliminated

Recommended: Idea 1. Shall I proceed with implementation?

If AUTO_PROCEED=false: Wait for user confirmation before continuing. The user may:

• Approve the idea → proceed to Stage 2. /experiment-bridge reads refine-logs/EXPERIMENT_PLAN.md already generated by /idea-discovery.
• Request changes (e.g., "combine Idea 1 and 3", "focus more on X") → update the idea prompt with user feedback, re-run /idea-discovery with refined constraints, and present again.
• Reject all ideas → collect feedback on what's missing, re-run Stage 1 with adjusted research direction. Repeat until the user commits to an idea.
• Stop here → save current state to idea-stage/IDEA_REPORT.md for future reference.

If AUTO_PROCEED=true: Present the top ideas, wait 10 seconds for user input. If no response, auto-select the #1 ranked idea (highest pilot signal + novelty confirmed) and proceed to Stage 2. Log: "AUTO_PROCEED: selected Idea 1 — [title]".

⚠️ This gate waits for user confirmation when AUTO_PROCEED=false. When true, it auto-proceeds after presenting results. The rest of the pipeline (Stages 2-3) is expensive (GPU time + multiple review rounds), so set AUTO_PROCEED=false if you want a final review checkpoint before committing GPU resources.

Stage 2: Experiment Bridge (Workflow 1.5)

Once the user confirms which idea to pursue, delegate implementation and deployment to /experiment-bridge:

/experiment-bridge "$CHOSEN_IDEA_TITLE" — code review: $CODE_REVIEW, base repo: $BASE_REPO, compact: $COMPACT

💡 Queue routing is automatic: /experiment-bridge Phase 4 routes each milestone by job count — ≤5 jobs → /run-experiment, ≥10 jobs or teacher→student phase dependencies → /experiment-queue (with OOM retry, wave gating, crash-safe state). No manual override is needed.

What this does (fully autonomous):

1. Parses refine-logs/EXPERIMENT_PLAN.md — extracts milestones, run order, compute budget
2. Implements experiment code — extends pilot to full scale, follows existing codebase conventions
3. Cross-model code review — GPT-5.4 xhigh reviews the implementation for logic bugs, incorrect metrics, and ground-truth misuse before any GPU time is spent
4. Sanity check — runs the smallest experiment first to verify the environment; auto-debugs failures (up to 3 attempts, with /codex:rescue fallback)
5. Deploys full experiments — auto-routes by job count (≤5 → /run-experiment, ≥10 → /experiment-queue with OOM retry, wave gating, crash-safe state)
6. Collects initial results — parses outputs, updates refine-logs/EXPERIMENT_TRACKER.md, runs /training-check if W&B is configured
7. Auto-plans ablations via /ablation-planner if main results are positive

Output:

• refine-logs/EXPERIMENT_RESULTS.md — structured results by milestone
• refine-logs/EXPERIMENT_TRACKER.md — updated run-by-run status
• EXPERIMENT_LOG.md (when COMPACT=true) — session-recovery-friendly log

Monitor progress (while experiments run):

/monitor-experiment [server]

Wait for /experiment-bridge to complete and report its handoff summary before proceeding.

Stage 3: Auto Review Loop (Workflow 2)

Once initial results are in, start the autonomous improvement loop:

/auto-review-loop "$ARGUMENTS — [chosen idea title], difficulty: $REVIEWER_DIFFICULTY"

What this does (up to 4 rounds):

1. GPT-5.4 xhigh reviews the work (score, weaknesses, minimum fixes)
2. Claude Code implements fixes (code changes, new e