Lecture 13: Reliability

reliability

(1) the consistency of a measurement, across various conditions; (2) freedom from error

reliability is related to

variance of true differences in scores and differences due to error

systematic error

constant; by design of measure/assessment

unsystematic error

random; lowers test reliability (e.g., motivation, admin conditions) “social desirability”

test reliability

The ratio of “true variance” to “observed variance”

reliability coefficients range from

0.00-1.00 (the higher the coefficient, the higher reliability)

test-retest coefficient

Administer the same test, to the same group, on 2 occasions (several days/weeks apart); Compute correlation coefficient from both sets of scores

benefits of test-retest coefficient

accounts for differences due to various conditions associated with 2 occasions (unsystematic error); somewhat practical, inexpensive

cons of test-retest coefficient

same test items, so no control over design error (systematic error)

parallel forms

Administering 2 versions (forms) of a test, both equivalent (in terms of quality, form, degree of difficulty, etc.), but not the same items; Compute correlation coefficient from both (parallel) forms

benefits of parallel forms

accounts for differences due to various test items (systematic error); Could be used in a way (e.g., one form given at one time, the other form a week later) to also account for error due to condition (unsystematic error)

cons of parallel forms

more involved (time, resources)

split-half method

Splitting a test’s scores in half and creating 2 sets of scores (i.e., 2 Ms and SDs)

benefits of split-half method

convenient and efficient

cons of split-half method

inflated reliability values

Cronbach’s Alpha

A formula for estimating correlations among all test items

benefits of Cronbah’s Alpha

convenient and practical; SPSS calculates; widely used and accepted in psych research

These coefficient values might reflect poor construct validity

the test is reliable but doesn’t measure the construct accurately