PSY 12: RELIABILITY

RELIABILITY

In psychometrics refers to consistency in measurement.In the psychometric sense it really only refers to something that is consistent—not necessarily consistently good or bad, but simply consistent

RELIABILITY COEFFFICIENT

a theoretical estimate of the correlation between actual scores of a psychometric test and the assumed true scoresIt is a proportion that indicates the ratio between the true score variance on a test and the total variance.

CLASSICAL TEST THEORY

a score on an ability test is presumed to reflect not only the test taker's true score on the ability being measured but also error

ERROR

refers to the component of the observed test score that does NOT have to do with the test taker's ability

MEASUREMENT ERROR

refers to the all of the factors associated with the process of measuring some variable, other than the variable being measured

RANDOM ERROR

source of error in measuring a targeted variable caused by unpredictable fluctuations and inconsistencies of other variables in the measurement process

SYSTEMATIC ERROR

source of error in measuring a variable that is typically constant or proportionate to what is presumed to be the true value of the variable being measured

TEST CONSTRUCTION

One source of variance during test construction is item sampling or content sampling

ITEM SAMPLING OR CONTENT

refer to variation among items within a test as well as to variation among items between tests

COEFFICIENT OF STABILITY

the estimate of test-retest reliability when the interval between testing is greater than six months

COEFFICIENT OF EQUIVALENCE

The degree of the relationship between various forms of a test can be evaluated by means of an alternate-forms or parallel-forms coefficient of reliability

SAMPLING ERROR

the extent to which the population of voters in the study actually was representative of voters in the election

METHODOLOGICAL ERROR

a source of error when collecting data of public opinion through survey and polls.

TEST RELIABILITY ESTIMATES

an estimate of reliability obtained by correlating pairs of scores from the same people on two different administrations of the same test.

COEFFICIENT OF EQUIVALENCE

The degree of the relationship between various forms of a test can be evaluated by means of an alternate-forms or parallel-forms coefficient of reliability

PARALLEL FORMS OF A TEST

for each form of the test, the means and the variances of observed test scores are equal. In theory, the means of scores obtained on parallel forms correlate equally with the true score.

PARALLEL FORMS RELIABILITY

refers to an estimate of the extent to which item sampling and other errors have affected test scores on versions of the same test when, for each form of the test, the means and variances of observed test scores are equal.

ALTERNATE FORMS RELIABILITY

refers to an estimate of the extent to which these different forms of the same test have been affected by item sampling error, or other error.

SPLIT-HALF RELIABILITY ESTIMATES

obtained by correlating two pairs of scores obtained from equivalent halves of a single test administered once.

SPEARMAN-BROWN FORMULA

allows a test developer or user to estimate internal consistency reliability from a correlation of two halves of a test.

SPEARMAN-BROWN FORMULA

may be used to estimate the effect of the shortening on the test's reliability

SPEARMAN-BROWN FORMULA

used to determine the number of items needed to attain a desired level of reliability.

INTER-ITEM CONSISTENCY

refers to the degree of correlation among all items on a scale.

HOMOGENEITY

is the degree to which a test measures a single factor.

HETEROGENEITY

describes the degree to which a test measures different factors

KUDER-RICHARDSON FORMULA 20 OR KR-20

is the statistic of choice for determining the inter-item consistency of dichotomous items, primarily those items that can be scored right or wrong (such as multiple-choice items).

COEFFICIENT ALPHA OR COEFFIEINET a-20

The one variant of the KR-20 formula that has received the most acceptance is in widest use today is a statistic called _________________________________

COEFFICIENT ALPHA

Developed by Cronbach and subsequently elaborated on by others.

COEFFICIENT ALPHA

In contrast to KR-20, which is appropriately used only on tests with dichotomous items, ____________________ is appropriate for use on tests containing non dichotomous items.

INTER-SCORER RELIABILITY

is the degree of agreement or consistency between two or more scorers (or judges or raters).

POEWR TEST

time limit is long enough to allow testtakers to attempt all items, and if some items are difficult that no testtaker is able to obtain a perfect score.

SPEED TEST

generally contains items of uniform level of difficulty (typically or uniformly low) so that when given generous time limits, all testtakers should be able to complete all the test items correctly

CRITERION-REFERENCES TESTS

designed to provide an indication of where a testtaker stands with respect to some criterion such as an educational or vocational objective.