Ch. 5 Ways to operationalizing constructs, scales of measurement, reliability

3 common ways to operationalize a construct

1. Self-report measures

2. Observational/ behavioral measures

3. Physiological measures

When you operationalize a variable, it must have atleast 2 levels

1. levels of categorical variables —> Categories (nominal)

2. levels of quantitative variables —> meaningful numbers (Ordinal, scale)

Scales of Measurement: Nominal

1. Non-ordered categorical responses

2. Allow us to determine whether 2 individuals are different, but we cannot make quantitative comparisons

Scales of Measurements: Ordinal

1. Ordered categorical responses

2. We can determine whether 2 individuals are different and determine the direction of difference

3. We cannot determine the magnitude of the difference

Scales of Measurement: Interval

1. Numerical categories that are equally spaced

2. Can determine magnitude of difference

3. No absolute 0- cannot make ratio statements

Scales of Measurement: Ratio

1. Can determine magnitude of difference

2. Can make ratio statements because —> absolute 0

Reliability

1. if you measure something multiple times under the same conditions, you should get the same results each time. It’s about being consistent and dependable.

2. true score + measurement error = observed score

Assessing Reliability (all 3 use the correlation coefficient)

1. test- retest

2. internal consistency

3. Interrater

correlation coefficient (x)

indicative of strength (-1 to +1) and direction (+ or -;slope)

Test-Retest Reliability

Indicates that the scores on a test will be similar when participants complete the test more than once

A strong reliability coefficient for test-retest reliability is….

0.5

internal Consistency

tests relationships between scores on different items of a survey

Cronbach’s alpha

average correlation between scores on all pairs of items on survey

Split-half reliability

divide test in half and correlate scores from each half

what is considered strong for internal consistency and inter-rater reliability

0.7

Inter-Rater Reliability

1. Degree of agreement between observers who are measuring the same behaviors

2. can be assessed by examining the judgements/ratings by multiple observers

5 different types of validity (is the test measuring what it is intended to measure)

Face, content, criterion, convergent, discriminant

Face Validity

A measurement procedure superficially appears to measure what it claims to measure

Content Validity

1. How well do the items represent the entire universe of items?

2. construct, variable, concept

Criterion Validity

Does the measure under consideration associate with a concrete behavioral outcome that it should be associated with?

Convergent Validity

a measure should correlate more strongly with other measures of the same similar constructs

Discriminant Validity

A measure should not correlate strongly with other measures of unrelated constructs

Convergent and Discriminant validity

often evaluated together at patterns of correlations among self-report measures

Reliability and validity

1. Can be reliable without being valid

2. can’t be valid without being reliable