idea-discovery

Workflow 1: Idea Discovery Pipeline

Orchestrate a complete idea discovery workflow for: $ARGUMENTS

Overview

This skill chains sub-skills into a single automated pipeline:

/research-lit → /idea-creator → /novelty-check → /research-review → /research-refine-pipeline
  (survey)      (brainstorm)    (verify novel)    (critical feedback)  (refine method + plan experiments)

Each phase builds on the previous one's output. The final deliverables are a validated idea-stage/IDEA_REPORT.md with ranked ideas, plus a refined proposal (refine-logs/FINAL_PROPOSAL.md) and experiment plan (refine-logs/EXPERIMENT_PLAN.md) for the top idea.

Constants

• PILOT_MAX_HOURS = 2 — Skip any pilot experiment estimated to take > 2 hours per GPU. Flag as "needs manual pilot" in the report.
• PILOT_TIMEOUT_HOURS = 3 — Hard timeout: kill any running pilot that exceeds 3 hours. Collect partial results if available.
• MAX_PILOT_IDEAS = 3 — Run pilots for at most 3 top ideas in parallel. Additional ideas are validated on paper only.
• MAX_TOTAL_GPU_HOURS = 8 — Total GPU budget across all pilots. If exceeded, skip remaining pilots and note in report.
• AUTO_PROCEED = true — If user doesn't respond at a checkpoint, automatically proceed with the best option after presenting results. Set to false to always wait for explicit user confirmation.
• REVIEWER_MODEL = gpt-5.5 — Model used via a secondary Codex agent. Must be an OpenAI model (e.g., gpt-5.5, o3, gpt-4o). Passed to sub-skills.
• ARXIV_DOWNLOAD = false — When true, /research-lit downloads the top relevant arXiv PDFs during Phase 1. When false (default), only fetches metadata. Passed through to /research-lit.
• COMPACT = false — When true, generate compact summary files for short-context sessions and downstream skills. Writes idea-stage/IDEA_CANDIDATES.md.
• OUTPUT_DIR = idea-stage/ — All idea-stage outputs go here. Create the directory if it doesn't exist.
• REF_PAPER = false — Reference paper to base ideas on. Accepts a local PDF path, arXiv URL, or paper URL. When set, summarize it first and use it as idea-generation context.
• RENDER_HTML = true — When true (default), auto-render idea-stage/IDEA_REPORT.md to HTML at workflow end via /render-html. Uses --no-review (source already passed novelty + cross-model review during Phase 3). Set false to skip, or pass — render html: false.

💡 These are defaults. Override by telling the skill, e.g., /idea-discovery "topic" — ref paper: https://arxiv.org/abs/2406.04329 or /idea-discovery "topic" — compact: true.

Pipeline

Phase 0: Load Research Brief (if available)

Before starting any other phase, check for a detailed research brief in the project:

1. Look for RESEARCH_BRIEF.md in the project root or a path passed in $ARGUMENTS.
2. If found, read it and extract:
3. problem statement and context
4. constraints: compute, data, timeline, venue
5. what the user already tried and what did not work
6. domain knowledge and non-goals
7. existing results, if any
8. Use this as the primary context for all subsequent phases; it replaces the one-line prompt when more specific.
9. If both RESEARCH_BRIEF.md and one-line $ARGUMENTS exist, merge them: the brief has priority for details, and the argument sets the direction.

If no brief exists, proceed normally with $ARGUMENTS as the research direction.

Recommended template:

# Research Brief

## Problem Statement
[What problem are we trying to solve?]

## Context
[Relevant field, current approach, why this matters]

## Constraints
- Compute:
- Data:
- Timeline:
- Target venue:

## What We Already Tried
- [attempt] -> [outcome]

## Non-Goals
- [what not to pursue]

Phase 0.5: Reference Paper Summary (when REF_PAPER is set)

Skip entirely if REF_PAPER is false.

Summarize the reference paper before searching the literature:

1. If arXiv URL — invoke /arxiv "ARXIV_ID" — download to fetch the PDF, then read the first 5 pages.
2. If local PDF path — read the PDF directly, focusing on the title, abstract, introduction, and method overview.
3. If other URL — fetch the content and extract the method, results, and limitations.
4. Generate idea-stage/REF_PAPER_SUMMARY.md using this template:

# Reference Paper Summary

## What They Did
[2-3 sentences: core method and contribution]

## Key Results
[Main quantitative findings]

## Limitations & Open Questions
[Acknowledged weaknesses, missing experiments, future work]

## Potential Improvement Directions
[Concrete ways to extend, challenge, or improve the paper]

## Codebase
[If `base repo` is set: link to the repo and identify relevant entry points]

Use idea-stage/REF_PAPER_SUMMARY.md as additional context in both Phase 1 and Phase 2.

Phase 1: Literature Survey

Invoke /research-lit to map the research landscape:

/research-lit "$ARGUMENTS"

What this does:

• Search arXiv, Google Scholar, Semantic Scholar for recent papers
• Build a landscape map: sub-directions, approaches, open problems
• Identify structural gaps and recurring limitations
• Output a literature summary (saved to working notes)

🚦 Checkpoint: Present the landscape summary to the user. Ask:

📚 Literature survey complete. Here's what I found:
- [key findings, gaps, open problems]

Does this match your understanding? Should I adjust the scope before generating ideas?
(If no response, I'll proceed with the top-ranked direction.)

• User approves (or no response + AUTO_PROCEED=true) → proceed to Phase 2 with best direction.
• User requests changes (e.g., "focus more on X", "ignore Y", "too broad") → refine the search with updated queries, re-run /research-lit with adjusted scope, and present again. Repeat until the user is satisfied.

Phase 2: Idea Generation + Filtering + Pilots

Invoke /idea-creator with the landscape context and idea-stage/REF_PAPER_SUMMARY.md if available:

/idea-creator "$ARGUMENTS"

What this does:

• If idea-stage/REF_PAPER_SUMMARY.md exists, include it as context so ideas explicitly build on, improve, or extend the reference paper
• Brainstorm 8-12 concrete ideas via GPT-5.4 xhigh
• Filter by feasibility, compute cost, quick novelty search
• Deep validate top ideas (full novelty check + devil's advocate)
• Run parallel pilot experiments on available GPUs (top 2-3 ideas)
• Rank by empirical signal
• Output idea-stage/IDEA_REPORT.md

🚦 Checkpoint: Present idea-stage/IDEA_REPORT.md ranked ideas to the user. Ask:

💡 Generated X ideas, filtered to Y, piloted Z. Top results:

1. [Idea 1] — Pilot: POSITIVE (+X%)
2. [Idea 2] — Pilot: WEAK POSITIVE (+Y%)
3. [Idea 3] — Pilot: NEGATIVE, eliminated

Which ideas should I validate further? Or should I regenerate with different constraints?
(If no response, I'll proceed with the top-ranked ideas.)

• User picks ideas (or no response + AUTO_PROCEED=true) → proceed to Phase 3 with top-ranked ideas.
• User unhappy with all ideas → collect feedback ("what's missing?", "what direction do you prefer?"), update the prompt with user's constraints, and re-run Phase 2 (idea generation). Repeat until the user selects at least 1 idea.
• User wants to adjust scope → go back to Phase 1 with refined direction.

Phase 3: Deep Novelty Verification

For each top idea (positive pilot signal), run a thorough novelty check:

/novelty-check "[top idea 1 description]"
/novelty-check "[top idea 2 description]"

What this does:

• Multi-source literature search (arXiv, Scholar, Semantic Scholar)
• Cross-verify with GPT-5.4 xhigh
• Check for concurrent work (last 3-6 months)
• Identify closest existing work and differentiation points

Update idea-stage/IDEA_REPORT.md with deep novelty results. Eliminate any idea that turns out to be already published.

Phase 4: External Critical Review

Fo