Experiments

Why experiments?

We often want generalizable, causal theories of human behavior. The central tool for building them is the experiment: a study in which the researcher manipulates some aspect of participants' experience and then measures an outcome. Experiments are "unreasonably effective" because random assignment lets us isolate a specific causal effect from the noise of everything else.

This approach's guiding thesis:

Experiments are intended to make maximally unbiased, generalizable, and precise estimates of specific causal effects.

Observational studies don't reveal causality

In an observational study we measure variables as they naturally occur. Two problems block causal claims:

Directionality — if A and B are correlated, we don't know whether A → B, B → A, or both. Does anxiety cause poor sleep, or poor sleep cause anxiety?
Confounding — a third variable C may drive both A and B, producing a spurious association. People who drink coffee may also smoke more; "coffee causes heart disease" could be confounded by smoking.

Observation can describe and predict, but it cannot, by itself, establish that changing X causes Y.

Experiments help us answer causal questions

The trick is random assignment of participants to conditions. Because assignment is random, the manipulated variable becomes the only systematic difference between groups (on average). Any difference in the outcome can then be attributed to the manipulation.

We can't hold people constant. Individuals differ in countless ways. Randomization balances those differences across conditions—no need to measure or match every confound.
Unhappy randomization? Randomization works on average and in expectation. Very small samples can still be unbalanced by chance, which is why sampling (see Sampling) and design (see Design) matter.

Generalizability

Even a perfectly randomized experiment only estimates an effect in the population and setting actually sampled. A causal effect estimated in one lab, with one stimulus set, may not hold elsewhere. Generalizability must be considered at every stage—stimulus selection, sampling, analysis, and reporting.

Anatomy of a randomized experiment

A clean experiment has three ingredients:

A manipulated independent variable (the cause you control).
A measured outcome variable (the effect you observe).
Random assignment of participants to levels of the manipulation.

Contrast this with "an experiment with very unclear causal inferences": if the manipulation is confounded with another change (e.g., condition A uses a different stimulus and a different instruction), you cannot tell which factor produced the outcome. Disentangling cause from confound is the job of Design.

Key terms

Causal effect — the change in an outcome produced by changing a factor.
Confound — a variable that varies with the manipulation and could explain the outcome.
Random assignment — each participant has a known chance of receiving each condition.