Scientific Narrative Builder

Related skills. This skill composes with hypothesis-building (where the "Why" funnel lands as a falsifiable "If-Then" with a named estimand), methods-reporting (where the design the narrative promises gets documented to APSA/JARS/DA-RT standards), and pre-registration-writing (where the narrative locks in before data collection).

Workflow

Author or audit an introduction by walking these five steps in order. Each step has a specific output; do not proceed until the prior step's output exists in the draft. See reference/example-funnel.md for a worked example.

Step 1 — Why: establish the substantive motivation

Output: one or two paragraphs naming the real-world social/political tension the study addresses, the specific "invisible" the design will reveal, and the analytical joint where existing theory is silent or conflicted. Do not frame the motivation as a gap-in-literature; frame it as a stake in the world.

• Identify the "Invisible": Treat surveys and experiments not just as data collection, but as a process to uncover "invisible factors", specifically perceptions, knowledge, beliefs, and reasoning that administrative data cannot capture.
• Carve the Analytical Joints: Do not settle for a broad topic. Identify the specific analytical joint or tension where existing theory is silent or conflicted -- "creativity in design is grounded in a capacity to carve a problem at its analytical joints" (Sniderman 2018). The introduction must articulate the specific "why" of the phenomenon before proposing the "how" of the experiment.
• The ASK Framework: Druckman (2022) identifies three pathways to research questions: Assessing (observing the world and noticing puzzles), Socializing (conversations with colleagues, students, and practitioners), and Kaput (learning from failed experiments -- "the first thing one should do when an experiment fails is to ask why it failed"). The narrative should be able to trace the question's origin to one of these pathways.
• Resist the "Methods-Driven" Temptation: Ensure the research question dictates the method, not vice versa. Explicitly defend why an experiment is the necessary tool for this specific question. "It is crucial to not jump to designs for their novelty, but only to turn to them when they offer an advantage over what could otherwise have been done" (Druckman 2022). Note that "writing your survey questions is already part of the analysis stage" (Stantcheva 2023) -- question design choices are analytical decisions, not logistical ones, and the narrative should convey this.
• Contextualization: Situate the study within a "broader account of politics" or social life. The narrative must move from the general social importance to the specific theoretical gap. For policy-relevant studies, explicitly connect the research question to the target policy environment and the populations affected.
• Multi-Level Theoretical Framing: When a study includes experiments at different levels of analysis (e.g., individual-level and institutional-level), the introduction must establish the theoretical bridge between levels. State explicitly how reasoning at the micro level (e.g., fairness judgments about individual immigrants) connects to reasoning at the macro level (e.g., legitimacy judgments about governance decisions). The bridge should be conceptual, not just methodological -- explain why the same theoretical mechanism should operate at both levels.

Step 2 — Evidence audit: what the literature actually establishes (and where it fails)

Output: a literature review section that states, for each prior finding cited, (a) what observable implication it establishes, (b) what design features limit its generalizability, and (c) what remains unresolved. The audit must not collapse into a chronological "studies find X" recital.

• Systematic Evidence Assembly: Move beyond "selective storytelling." Favor the assembly of estimates from meta-analyses or registries. If these do not exist, explicitly document the search and inclusion criteria to avoid publication bias. Study registries (EGAP, AEA, OSF) make the existence of studies visible even when their results go unpublished -- reference this universe of studies when available (Christensen et al. 2019).
• Publication Bias Acknowledgment: When reviewing prior findings, explicitly note where the published literature may overstate effect sizes due to the "file drawer problem" -- the systematic non-publication of null results. If a registered universe of studies exists, the narrative should reference it. Note where prior effect sizes may be inflated by publication bias, and frame the current study's power analysis conservatively (Druckman 2022; Lakens 2025).
• Adjudicating Theories: Clarify if the literature review is setting up a "Fact Searching" mission (estimating a causal effect) or a "Theory Testing" mission (adjudicating between competing psychological or social mechanisms). The stronger narrative frame is "explication, not demonstration" -- the goal is to explain why effects occur, not merely to show they exist (Sniderman 2018).
• The Three Modesties and Sample Modesty: Acknowledge Sniderman's (2018) three limitations of survey experiments -- "modesty of treatment, modesty of scale, and modesty of measurement" (single dependent variable, single indicator, short duration) -- and separately acknowledge Mutz's (2011) point on sample modesty: an accessible population may not match the target population ("college students may not be people"). The narrative should also acknowledge a fourth modesty: the current design relative to the full theoretical question. Frame the study as one link in a progression of trials -- what Sniderman (2018) calls a "chain of discovery" extending and cross-validating lines of research -- rather than a one-off decisive demonstration.
• Replication-Extension Strategy: When building on prior work, frame the relationship as a "replication-extension" -- the current study replicates key features of an established design while extending it to address new questions or boundary conditions. Prior studies are not just evidence to cite but designs to build upon (Druckman 2022).
• Convenience Sample Limitations: When reviewing findings from prior convenience-sample experiments (e.g., student subject pools), explicitly acknowledge that treatment effects may be heterogeneous across the population. Effects observed in narrow samples may not replicate in broader populations, and the direction or magnitude may change (Mutz 2011, citing Rashotte and Webster).
• Distortions in the Evidence Base: Rather than attributing motive ("bad faith"), target observable behaviors that bias the literature. Note where cited findings may be distorted by specification searching, selective reporting, researcher degrees of freedom, or publication incentives (Christensen, Freese, and Miguel 2019; Simmons, Nelson, and Simonsohn 2011; Wicherts et al. 2016). Where an original study's analytical choices are not pre-registered, treat its reported effect size as an upper bound, and calibrate the current study's power and severity framing accordingly.

Step 3 — Funnel: narrow Why to If-Then

Output: a transition section that moves from the general theoretical claim (Step 1) through the specific counterfactual the design creates to the statistical hypothesis the data will test. The funnel must name the target population, the identifying variation, and the estimand before stating the hypothesis.

• The Funnel Structure: Organize the narrative into a "Why" funnel. Start with the general theory (the explanation of why things happen in the world) and narrow down to the specific hypothesis (the "If-Then" statement of what will be observed in this data).
• Defining the Target Population: Do not wait for the methods section to define the population. The introduction must specify the scope o