Measurements (Psch 242)

Objective measurements are not ___ related to the construct you’re interested in

Exclusively

Ex: IQ tests are an indicator of cognitive ability, but also level of education, motivation, test anxiety, and more!

What two methods are good to use for objective measurements? And why?

Both objective and subjective as it is good for establishing validity.

What are two sources of measurement error? And what are they?

Random error-inconsistent measurement procedures/unreliable instruments

Systematic error-measure does not capture construct of interest/contamination (participants aren’t honest)

Define reliability.

How consistent your measure is (absence of random error)

Define test-retest reliability

Consistency over time

Ex: Correlation between scores on test (Time 1) and retest (Time 2)

- Results should be similar if you measure the same thing the same way multiple times

Describe the perfect test-retest reliability.

Measurements don’t change between time 1 and time 2. Each person gives the same answer twice

Describe a fair-retest reliability.

People’s scores change a little from time 1 and time 2 (still similar)

Describe poor test-retest reliability.

People are changing a lot between time 1 and 2 (seemingly no relationship)

Define interrater reliability

Degree of agreement between two independent raters

- Correlation between (at least) two raters’coding

- If two people see the same thing, are they interpreting it in the same way

Define internal consistency reliability.

Degree to which the items of a measure reliably measure the same thing

Correlations between first and second set of questions (split-half)

What is cronbach’s alpha?

Average of all possible split-half correlations

- If two questions are trying to measure they same thing, they should get similar results!

- Ex. “Do you feel anxious?” should get similar answers to “Do you feel nervous?”

When does a measurement have high construct validity?

When it measures what it is supposed to measure.

When does a measure have high convergent validity?

When it is strongly (positively) associated with existing (validated) measures of the same thing.

Discriminant validity

What are two types of criterion validity?

Predictive and criterion validity

What does predictive validity ask?

Does the measure predict construct-relevant behaviors/outcomes?

Ex: The ACT should predict performance in college

- A measure of trait self-control should predict people’s ability to resist temptations

What does criterion validity ask?

Does it match the gold-standard measurement?

Ex: Does your new heart monitor agree with existing heart monitors?

- Does your new short version measure of anxiety give the same results as the complete and validated test?

What is effective range?

The range (low to high) within which a measure gives accurate measurements of the variable (that you’re measuring)

What is the floor effect?

Everyone/almost everyone is at the bottom of a scale

• The range at the bottom isn’t sufficient to pick up variation

• Ex: • How do you feel about having to walk in heavy rain

  without an umbrella on a cold day?

  1. Very Positive

  2. Positive

  3. Slightly positive

  4. Neutral  This is the floor

What is the ceiling effect?

Everyone/almost everyone is at the top of a scale

• The range is insufficient

• Ex: How tall are you?

  • 4’7 – 5’0  This is the ceiling

  • 4’0 – 4’6

  • 3’7 – 4’0

  • 3’0 – 3’

How can you ensure your measure has the right range?

Look at past studies and pilot test (beta test using a smaller sample to ensure sufficient variation)

Define variable.

Something that can vary

Ex: height, nationality, etc

Define value.

A number representing a certain level of the variable

Ex: Height: 5’7, 6’2, etc…

Define score.

A specific value for a given person or thing.

Ex: my score on the variable “height” is 5’9

What two things should be considered when creating or using a measure?

1. Properties of numbers

2. Properties of the thing you’re measuring

What are the properties of numbers?

Some variables have all these properties, others don’t

What are the properties of variables?

They are measured on different scales.

What are categorical variables?