Chapter 5

Measurement Concepts

Reliability

Consistency and stability of a measure—it performs the same way under consistent conditions

Measurement formula

Observed score = True score + Systematic error + random error

• Random error (e.g., mood, distractions) reduces reliability

• Systematic error - consistently skews scores (e.g., watch running fast) but doesn’t necessarily reduce reliability

Types of reliability

• Test-retest

• Equivalent

• Internal consistency

• Inter-rater

• Key insight: Consistency does not = accuracy, A measure can reliably give the same wrong result

Test-restest reliability

Consistency across time. Good for traits, not for states

Equivalent (Alternate) Forms Reliability

Comparing different forms of the measure

Internal consistnecy

How well items within a scale correlate (often via split-half of Cronbach’s alpha)

Inter-rater reliability

consistency across different observers

Construct validity

Does the operational definition truly represent the theoretical construct?

• face

• content

• convergent

• discriminant (divergent)

• criterion

  • Concurrent

  • Predictive

Face validity

The measure appears appropriate on its face

Content validity

The measure adequately covers the full range of the range of the construct

Convergent validity

does the measure correlate with others that it theoretically should?

Discriminant (Divergent) validity

Does it not correlate with measures it shouldn’t?

Criterion validity

How well does the measure predict or correlate with a relevant outcome?

• Concurrent: correlation with criterion at the same time

• Predictive validity: prediction of future outcomes

Reactivity

Occurs when awareness of being measured influences participants’ behavior (e.g., knowing they’re being observed)

Unobtrusive

(Nonreactive) measures help minimize this effect

Measurement Scales

• Nominal scale

• Ordinal scale

• Interval scale

• Ratio scale

Nominal scale

Categorical labels (e.g., gender, yes/no)—no numeric meaning or order

Ordinal scale

Ranking or order without equal intervals (e.g., 1st, 2nd, 3rd)

Interval scale

Ordered with consistent intervals but no absolute zero (e.g., IQ scores, temperature in Celsius)

Ratio scale

Interval scale with a meaningful zero point (e.g., reaction time)