Design

Experimental designs

• A two-factor experiment — cross two independent variables to estimate main effects and their interaction.
• Generalized factorial designs — combine many factors; estimation strategies balance the number of conditions against power.
• Between- vs. within-participant designs — between-subjects avoids carryover but needs more participants; within-subjects is efficient but risks practice/order effects.
• Repeated measures and experimental items — participants respond to many items; both participant and item are sources of variance (see Models).
• Stimulus-specific effects — a result driven by one stimulus set may not generalize; vary stimuli to check.
• Discrete and continuous manipulations — categorical vs. graded manipulations (e.g., titrated designs that vary intensity).

Choosing your manipulation

• Internal validity threats: confounding — if the manipulation is bundled with another change, you can't tell which caused the outcome (see Experiments).
• Internal validity threats: placebo, demand, expectancy — participants' beliefs about the condition can produce effects unrelated to the manipulation. Blinding and placebo controls help.
• External validity of manipulations — a manipulation that works in the lab may not map onto the real-world construct. Ask whether the manipulation is a faithful instance of the theory.

Key terms

• Factorial design — an experiment combining two or more independent variables.
• Within-participant — each person experiences all levels of a factor.
• Internal validity — the extent to which a causal claim is warranted within the study.

Further reading

• Previous: Measurement · Next: Sampling · Hub: Experimentology
• The cause you manipulate: Experiments
• Who you test: Sampling
• Experimental Design — the general design guide.
• Independent Variable and Dependent Variable — the manipulated cause and measured effect.
• Control Group — the counterfactual.
• Stroop Effect, Go/No-Go Task, Stop-Signal Task — paradigms embodying these design choices.
• Eriksen Flanker Task and Implicit Association Task — interference and implicit-measure paradigms built on these design principles.
• Attention Network Task — a factorial combination of cueing and flanker factors measuring separate attentional networks.
• Tower of London and Choice Reaction Time — problem difficulty and number-of-alternatives as manipulated independent variables.

References

• Fisher, R. A. (1935). The Design of Experiments. Oliver & Boyd.