Reliability vs Validity Practice

Practice distinguishing reliability (consistent, stable results) from validity (measuring what you intend to measure) in psychology measurement contexts.

What is the difference?

Reliability is about consistency: would the same measure give the same result again — across time, across raters, or across similar items? It answers the question 'is this measure stable?'

Validity is about accuracy: does the measure actually capture the concept it claims to measure? It answers 'are we measuring the right thing?'

A measure can be reliable but not valid. A badly calibrated scale is perfectly consistent (reliable) — it always shows the same weight — but if it is off by 10 kg, it is not valid. Validity is the more fundamental question.

How to practice

Follow these steps:

  1. Read the example and ask: is the concern about consistency/repeatability (reliability) or about measuring the true concept (validity)?
  2. Remember the rule: same results again → reliability. Actually measuring the right thing → validity.
  3. Choose your answer, read the explanation, and continue.

Worked example

A researcher checks whether a new anxiety scale agrees with clinical interviews.

Agreement with an established criterion (clinical diagnosis) shows the scale measures what it claims. → Validity. But if the same person takes the scale twice a week apart and gets nearly identical scores, that consistency is about → Reliability.

Frequently Asked Questions

Have another question? Let us know.

Can a measure be reliable but not valid?

Yes — this is the classic example. A scale that is consistently off by 10 kg is highly reliable (same result every time) but not valid (it doesn't show true weight). Reliability is necessary for validity but does not guarantee it.

What are the main types of reliability?

Test-retest reliability (same result over time), inter-rater reliability (agreement between observers), and internal consistency (agreement across items, e.g., split-half or Cronbach's alpha). All answer 'is the measure consistent?'

What are the main types of validity?

Content validity (items cover the concept), criterion validity (predicts or correlates with an outcome, e.g., predictive or concurrent), and construct validity (measures the right underlying concept, including convergent and discriminant validity).

Which matters more, reliability or validity?

Validity is more fundamental — a reliable measure of the wrong thing is worse than useless. However, low reliability caps validity: if a measure is noisy, it can never be highly valid. In practice, good psychometrics report and improve both.

Buy me a coffee