PSYC3020 L1 The Scope of Measurement Lecture questions

The psychometric concepts of reliability and validity are not specific to psychological measurement.

True. Applies to all measurements domains.

In psychology, we always use standardised scores when interpreting behavioural measures.

False. Sometimes the raw score is meaningful eg. reaction time

The z score is an example of a standardised unit of measurement.

True

Jamovi is a software package for creating surveys.

False. Qualtrics is for this course

Qualtrics is a spreadsheeting software package.

False. It's a survey package

Create your own example of a standardisation problem.

A parent uses the back of their hand to measure if a child has a temperature because they can't find a thermometer.

Give three examples of standardized units of measurement (any domain).

Meter, BPM, decibels

What's a raw score?

Direct results from a test before any further standardisation is carried out

Create your own example to explain the psychometric concept of reliability.

If you can get consistent results from your measure when measuring the same thing, then it is reliable

Give your own example of a situation when we would want to consider the reliability of a measurement (any domain).

Does a thermometer give you a reliable result (consistent temperature) when you keep measuring 3 times in a result?

Create your own example to explain how we could evaluate the reliability of a behavioural measure.

Does a test for aggression give the same score when measured on the same participant multiple times?

Create your own example to explain the psychometric concept of validity.

Do personality test actually test personality?

Give your own example of a situation involving the validity of a measurement (any domain).

Does a thermometer actually test the temperature of the part of the body you want to test.

Create your own example to explain how we could evaluate the validity of a behavioural measure.

Does the aggression test actually test aggression? Will it predict who will display more incidences of aggression?

Describe the typical steps involved in creating a behavioural or psychological measure.

1. Define the construct (What are you measuring? Why? How?).

2. Create materials (e.g., survey items).

3. Test for:

- Standardisation

- Reliability

- Validity

- Item quality

4. Refine based on data.

5. Release and use the measure.

What is Jamovi?

A statistical analysis package

What is Qualtrics?

A survey creating package

Imagine you run a survey examining the relationship between scores from a speed questionnaire with self-reported crash involvement. Contrary to previous research, you find no significant correlation. Describe five reasons that could account for this.

1. Lacks validity (self-report may not reflect true behaviour)

2. Sample size not big enough to detect significance

3. Not enough drivers in sample (low variability in participants)

4. Other important predictive variables were not included in survey

5. Lacks reliability (crash measure may be too vague)

Explain how we could use a relevant behavioural measure to predict a driver's crash risk.

We could measure many variables that have been shown to correlate with crashes (not just speed). e.g. alcohol consumption, sleep, personality traits (e.g. conscientiousness), etc.

Then run a regression model using these behavioural measures as predictors and crash risk as the outcome.

Explain how we could use a battery of relevant behavioural measures to predict a convicted criminal's likelihood of re-offending.

We could measure incidents since precious conviction, motivation to do better, employment status, family history, location of residence, etc.

Explain how we could use a battery of relevant behavioural measures to predict someone's likelihood of maintaining a particular physical exercise regime.

Measure engagement in previous regimes, motivation, previous successes, weight, addictions, medication, time available to practice regime.

Choose two methods of measuring behaviour and/or psychological traits/states. Outline the advantages and disadvantages of each.

- Survey (Pros: cheap, easy to get large sample size; cons: self-report can be unreliable)

- Observation (pros: more objective cons: more expensive)

Describe two measurement-related tasks that potential employers might ask PSYC3020 graduates to do.

- Design and validate a psychological survey instrument

- Conduct statistical analysis to evaluate reliability and validity of collected data