research-methods-psych

Research Methods in Psychology

Psychology's claim to be a science rests on its methods. Unlike philosophy or folk wisdom, psychology makes empirical claims testable through systematic observation and experimentation. This skill covers the experimental method as applied in psychological research, the ethical framework governing human subjects research, the statistical tools used to evaluate evidence, and the ongoing replication crisis that has forced the field to confront its methodological shortcomings.

Agent affinity: kahneman (research design, statistical thinking), james (methodological pragmatism, what constitutes evidence)

Concept IDs: psych-perception-construction, psych-cognitive-biases, psych-social-influence, psych-learning-theory

Research Methods at a Glance

| # | Domain | Core Question | Key Concepts | |---|---|---|---| | 1 | Experimental design | How do we establish causation? | IV/DV, random assignment, control, confounds | | 2 | Non-experimental methods | How do we study what we cannot manipulate? | Correlation, observation, case studies, surveys | | 3 | Research ethics | How do we protect participants? | Informed consent, deception, debriefing, IRB | | 4 | Statistics in psychology | How do we evaluate evidence? | NHST, effect size, confidence intervals, power | | 5 | The replication crisis | Can we trust published findings? | Publication bias, p-hacking, preregistration |

Domain 1 -- Experimental Design

The logic of experimentation

An experiment establishes causation by manipulating one variable (the independent variable, IV) while holding all other variables constant, and measuring the effect on another variable (the dependent variable, DV). Random assignment to conditions ensures that participant characteristics are distributed evenly across groups, eliminating systematic confounds.

Key elements

• Independent variable (IV) -- the factor the researcher manipulates. Must have at least two levels (e.g., treatment vs. control).
• Dependent variable (DV) -- the outcome measured. Must be operationally defined (e.g., "anxiety" measured by the Beck Anxiety Inventory score, not a vague assessment).
• Random assignment -- each participant has an equal chance of being assigned to any condition. This is what distinguishes a true experiment from a quasi-experiment.
• Control condition -- a comparison group that does not receive the treatment. May be no-treatment, waitlist, active control (alternative treatment), or placebo.
• Confound -- a variable that varies systematically with the IV, making it impossible to determine which caused the effect on the DV.

Between-subjects vs. within-subjects

| Design | Each participant | Advantage | Disadvantage | |---|---|---|---| | Between-subjects | Experiences one condition | No order effects, no demand characteristics from multiple testing | Needs more participants, individual differences add noise | | Within-subjects (repeated measures) | Experiences all conditions | Each participant serves as their own control, more statistical power | Order effects, practice effects, fatigue | | Mixed | Between on one IV, within on another | Combines advantages | Complexity in analysis and interpretation |

Counterbalancing (varying the order of conditions across participants) controls for order effects in within-subjects designs.

Factorial designs

When two or more IVs are crossed (every level of each IV is combined with every level of every other IV), the design is factorial. A 2x3 factorial has 6 conditions. Factorial designs reveal main effects (the effect of each IV averaging over others) and interactions (the effect of one IV depends on the level of another). Interactions are often more theoretically interesting than main effects.

Validity

• Internal validity -- confidence that the IV caused the change in the DV. Threatened by confounds, selection bias, maturation, history, and attrition.
• External validity -- generalizability to other populations, settings, and times. Threatened by non-representative samples, artificial laboratory settings, and demand characteristics.
• Construct validity -- the degree to which the IV and DV actually measure the constructs of interest.
• Statistical conclusion validity -- appropriate use of statistical tests, adequate sample size, and correct interpretation of results.

Domain 2 -- Non-Experimental Methods

Correlational research

Measures the relationship between two variables without manipulating either. Correlation does not establish causation because of the third-variable problem (an unmeasured variable may cause both) and the directionality problem (does A cause B or B cause A?). Correlation coefficients range from -1 to +1; the sign indicates direction, the magnitude indicates strength.

Observational methods

• Naturalistic observation -- observing behavior in its natural setting without intervention. High ecological validity, low internal validity.
• Structured observation -- observing behavior in a controlled setting with standardized procedures (e.g., Ainsworth's Strange Situation).
• Participant observation -- the researcher joins the group being studied.

Case studies

Intensive investigation of a single individual or small group. Invaluable for rare conditions (H.M., Phineas Gage, Genie) and for generating hypotheses, but cannot establish causation or generalize to populations.

Surveys

Self-report measures administered to large samples. Efficient for gathering data on attitudes, beliefs, and behaviors. Vulnerable to social desirability bias, acquiescence bias, and poorly worded questions.

Domain 3 -- Research Ethics

Historical catalysts

• Nuremberg Code (1947) -- established voluntary consent as essential, in response to Nazi medical experiments.
• Tuskegee Syphilis Study (1932-1972) -- untreated Black men with syphilis were studied for 40 years without informed consent. Led directly to the Belmont Report and modern IRB requirements.
• Milgram (1963) and Zimbardo (1971) -- raised questions about psychological harm from research participation.

APA Ethical Principles

The American Psychological Association's Ethical Principles (2017) include:

1. Beneficence and nonmaleficence -- maximize benefits, minimize harm
2. Fidelity and responsibility -- establish trust, accept responsibility
3. Integrity -- promote accuracy, honesty, truthfulness
4. Justice -- fair and equitable access to and benefit from research
5. Respect for people's rights and dignity -- protect autonomy, privacy, confidentiality

Key ethical requirements

• Informed consent -- participants must understand the study's purpose, procedures, risks, and their right to withdraw without penalty. Written consent is standard.
• Deception -- permitted only when (a) the study cannot be conducted without it, (b) the scientific value justifies it, and (c) participants are debriefed promptly. Milgram's experiments would require extensive justification today.
• Debriefing -- after participation, explain the study's true purpose and any deception. Address any distress caused.
• Institutional Review Board (IRB) -- independent committee that reviews research proposals for ethical compliance before data collection begins.
• Confidentiality -- participant data must be stored securely and reported in ways that prevent identification.

Vulnerable populations

Research with children, prisoners, cognitively impaired individuals, and other vulnerable populations requires additional safeguards: parental consent plus child assent, independent advocates, and heightened risk-benefit scrutiny.

Domain 4 -- Statistics in Psychology

Null Hypothesis Significance Testing (NHST)

The dominant (and controversial) inferential framework in psychology:

1. State a null hypothesis (H0: no effect/no difference) and an alternative hy