Sampling

Sampling theory

Classical probability sampling — every member of the population has a known, non-zero chance of selection, so estimates generalize.
Representative and stratified sampling — stratify on key variables (age, gender) to improve precision and ensure subgroups are represented.

Convenience samples, generalizability, and the WEIRD problem

Much psychology runs on WEIRD samples—Western, Educated, Industrialized, Rich, Democratic—which are oddly unrepresentative of humanity. Convenience samples (students, online panels) are cheap but limit claims about people in general. Reasons for hope: some effects are robust across cultures; ways forward include broader recruitment and explicit boundary claims.

Biases in the sampling process

Collider bias — conditioning on a variable influenced by both exposure and outcome creates a spurious association (a common, subtle trap).
Attrition bias — participants who drop out often differ systematically from those who stay, skewing results.

Sample size planning

Power analysis — choose N so the study has a high chance of detecting an effect of the size you care about (compute it with the Power & Sample Size Calculator).
Power analysis in practice — use pilot data or prior estimates of effect size; justify N before collecting.
Alternative approaches — precision-based planning (target a confidence-interval width) rather than a binary "significant" threshold.
Sample sizes for replication studies — replications often need larger N than the original to be informative. See Replication.

Key terms

Probability sampling — known selection probabilities enable generalization.
WEIRD — the unrepresentative populations psychology over-relies on.
Power — probability of detecting a real effect at a given sample size.