Experimental Psychology - Chapter 5 Identifying Good Measurement

Conceptual definition

a researcher's definition of a variable at the theoretical level

Self-report measure

a method of measuring a variable in which people answer questions about themselves in a questionnaire or interview

Observational measure

a method of measuring a variable by recording observable behaviors or physical traces of behaviors

Physiological measure

a method of measuring a variable by recording biological data

Categorical variable

A variable whose levels are categories (e.g., male and female). Also called nominal variable.

Quantitative variable

a variable whose values can be recorded as meaningful numbers

ordinal scale

a quantitative measurement scale whose levels represent a ranked order, in which it is unclear whether the distances between levels are equal

Interval scale

A quantitative measurement scale that has no "true zero," and in which the numerals represent equal intervals (distances) between levels (e.g., temperature in degrees).

Ratio scale

a quantitative scale of measurement in which the numerals have equal intervals and the value of zero truly means "nothing"

Reliability

The consistency of the results of a measure

Validity

The appropriateness of a conclusion or decision.

test-retest reliability

the consistency in results every time a measure is used

interrater reliability

The degree to which two or more coders or observers give consistent ratings of a set of targets.

internal reliability

in a measure that contains several items, the consistency in a pattern of answers, no matter how a question is phrased

correlation coefficient r

A single number, ranging from -1.0 to 1.0, that indicates the strength and direction of an association between two variables.

Slope direction

the upward, downward, or neutral slope of the cluster of data points in a scatterplot

Strength

a description of an association indicating how closely the data points in a scatterplot cluster along a line of best fit drawn through them

Average inter-item correlation (AIC)

a measure of internal reliability for a set of items; it is the mean of all possible correlations computed between each item and the others

Cronbach's alpha

a correlation-based statistic that measures a scale's internal reliability

face validity

the extent to which a measure is subjectively considered a plausible operationalization of the conceptual variable in question

content validity

the extent to which a measure captures all parts of a defined construct

Criterion validity

An empirical form of measurement validity that establishes the extent to which a measure is associated with a behavioral outcome with which it should be associated

known-groups paradigm

a method for establishing criterion validity, in which a researcher tests two or more groups, who are known to differ on the variable of interest, to ensure that they score differently on a measure of that variable

convergent validity

an empirical test of the extent to which a self-report measure correlates with other measures of a theoretically similar constructed

discriminant validity

an empirical test of the extent to which a self-report measure does not correlate strongly with measures of theoretically dissimilar constructs

Three Types of Reliability

Test-rested reliability

Interrater reliability

Internal reliability

Scatterplots can show Interrater agreement

if there is good Interrater reliability we should see we have a perfect diagonal line

Scatterplots can show Interrater disagreement

if there is no good Interrater reliability, we should see a lot of spread