research-idea-generator

Research Idea Generator

Generate a broad set of candidate research ideas, kill the weak ones internally, and present only the survivors. This skill is a generator with a built-in filter. The user sees 10 ranked ideas; the model generates 25-30 and discards the rest.

The vector-space principle

A good research idea is an arrow from the current literature toward reality. The larger the gap between what economists assume and how the world actually works, the more impactful the idea.

Three idea-source strategies, ranked by effectiveness:

1. Starting from practitioner reality (strongest): Real-world observations reveal where reality diverges from what economists assume. These observations mark where reality is, letting you draw the arrow from literature to reality.

1. Starting from a cool dataset (risky): A dataset generates many possible directions but no "hook" connecting to a specific literature-reality gap. Without a question, tinkering with data is unproductive. A dataset is a tool, not an idea.

1. Starting from the literature (weakest for generation): Reading papers and looking for extensions typically produces marginal additions. Literature knowledge is essential for positioning an idea, but is a poor source of ideas.

Phase 0: Collect user constraints

Before generating anything, collect and record these as hard filters on ranking. An idea that violates a hard constraint cannot rank in the top 5 regardless of its intellectual appeal.

| Constraint | Question to ask | How it binds | |---|---|---| | Data access | What databases, proprietary data, or institutional access do you have? | Ideas requiring unavailable data are capped at Medium feasibility | | Methods comfort | What empirical methods are you comfortable executing? (DiD, RDD, structural, NLP, etc.) | Ideas requiring unfamiliar methods are downranked unless a coauthor fills the gap | | Field comfort | What literatures do you know well enough to position a paper? | Ideas in unfamiliar fields require more literature work | | Journal ambition | Are you targeting a top generalist journal, a strong field journal, or any good outlet? | Determines contribution bar | | Coauthor skills | What complementary skills does a coauthor bring? | Relaxes method or field constraints | | Timeline | How soon do you need a working paper? | Long-horizon structural projects vs. quick empirical exercises |

If the user provides a topic and constraints in their initial message, proceed without asking. Fill defaults for anything not specified (assume broad data access, standard empirical methods, strong field-journal ambition, solo researcher, 12-month timeline).

Phase 1: Map the landscape

Check for existing data first: If output/paper_set.json exists, read it. If it contains papers relevant to the user's topic, use them as the starting point and supplement with targeted searches for gaps. This avoids redundant searching when the user has already run /lit-review or /lit-search.

Always open with the two-search pattern, then supplement:

1. Architecture search: search_papers (broad topic, sortBy: "citedByCount", matchCount: 15) — immediately see which papers define this field and its citation hierarchy.
2. Frontier search: search_papers (same topic, minYear: 2020, matchCount: 15) — what's happening now, who is active, what's the edge.
3. Survey search: search_papers (topic + "survey" or "review", matchCount: 10) — find survey papers that map open questions.

Use get_paper_details_batch on the top 5-10 results from the architecture search to read abstracts and understand what has been established.

Produce a 3-4 sentence internal landscape summary: what is settled, what is actively debated, where the frontier is moving. Use relative citation tiering (top 10% = foundational, next 30% = established, bottom 60% = emerging).

Save to shared files: Write all collected papers to output/paper_set.json (merge if exists). Append search queries to output/search_log.md.

Phase 2: Generate candidates (internal, not shown to user)

Apply all eleven lenses to the topic area. Generate 2-3 rough candidates per lens. Target 25-30 total candidates. Most will be discarded.

Stage A: Generative lenses (reality-first)

These lenses produce the raw material. Apply these first.

Lens 1: Practitioner-reality gap

Start from how real actors behave, and ask whether this conflicts with what economists assume. Use these specific elicitation prompts:

• What do practitioners optimize that theory says they should not?
• What variable do practitioners obsess over that papers ignore?
• What decision rule do people use in practice that would look irrational in a model?
• What do industry participants know that academics have not documented?

Search with search_papers for "[topic] survey evidence" or "[topic] practitioner" to find papers documenting real-world behavior. If the user provided anecdotes, use those as the starting point.

The test: can you name a specific assumption in the literature that conflicts with observed behavior? If yes, the idea is well-formed. If not, the observation is interesting but not yet an idea.

Lens 2: Cross-pollination

Borrow a well-established concept, method, or finding from a neighboring field (labor, IO, macro, behavioral, urban, computer science) and apply it where it has not been used. The imported concept must generate a genuinely different economic prediction in the new setting, not just "X but in field Y."

Lens 3: First principles / economic logic

Start from a fundamental friction (information asymmetry, moral hazard, search costs, limited attention, regulatory distortion, market segmentation) and derive a testable prediction. Then check whether anyone has tested it. The best first-principles ideas feel obvious in retrospect.

Lens 4: Unification

Multiple individually documented findings are manifestations of the same underlying economic force. The synthesis is the paper. List 3-5 well-known findings and ask: is there a single mechanism that explains all of them? The unification must generate a new testable prediction.

Lens 5: Policy / regulatory shock

A recent policy change creates a natural experiment. The policy is the instrument, not the contribution. The question must be bigger than the policy. Search for recent (past 3-5 years) shocks relevant to the topic.

Stage B: Refinement lenses (literature-positioned)

These lenses sharpen, reframe, or find angles on existing knowledge. Apply these second, using the landscape from Phase 1.

Lens 6: Literature gap

Find questions that survey papers flag as open. The gap must be economically meaningful, not just "nobody has run this regression." Caution: literature-driven ideas tend to produce marginal extensions. This lens should sharpen, not source.

Lens 7: Conflicting findings

Two credible papers reach opposite conclusions. The reconciliation is the idea. The moderating variable or identification flaw that explains the conflict becomes the contribution.

Lens 8: Mechanism decomposition

Take an established result and ask: what is the economic channel? Can you distinguish competing mechanisms? Many highly cited papers document effects without pinning down why. Include the measurement-error angle: does a canonical variable mismeasure the intended concept in a systematic way that changes interpretation? (Example: the literature thinks it measures financial constraint, but the proxy is contaminated by firm quality.)

Lens 9: Boundary conditions and equilibrium displacement

When does a canonical finding break down? The failure must map to a theory. Include the equilibrium-displacement angle: if agents adapt to the original finding, who absorbs the incidence or where does the effect move? In household finance, asset pricing, and real estate, strategic adjustment often produces deeper questions than simple treatment-effect framing