Psychological data

Experimental study

Experimental research: you manipulate the independent (predictor)

variable and record any changes in dependent variable.

• Example: giving people different amounts of alcohol before asking

them to do a driving test in a simulator.

Observational study

Observational research: you simply record both the dependent and

independent variables (no manipulation involved)

• Example: measuring the amount of alcohol consumed during

pregnancy and the birth weight of the child

• It is less easy to determine cause and effect in observational studies

Within subjects design

Within-subjects designs: all participants receive all levels of the

independent variable.

• Example: participants come in on 3 days and drink a whole pint on

one day, half a pint on the next and no alcohol on the final day

Between subjects design

Between-subjects designs: participants are separated into groups; each

group receive only one level of the independent variable.

• Example: participants are either given a full pint, a half pint or no

alcohol.

Mixed designs

In mixed designs some independent variables are within-subjects and

some are between.

Nominal data

Categories with no logical order, e.g., breakfast cereal, mode

of transport

• Can compare equality (=)

Ordinal data

Categories have an order but distance isn’t meaningful, e.g.,

highest qualification

• Compare equality, greater than/less than (=, >/<)

Interval data

Numbers represent values at regular intervals but with no

absolute zero, e.g., IQ scores/ year of birth

• Compare equality, greater than/less than and arithmetic (=,

>/<, +/-)

Ratio data

consistency across items (e.g.,

questions) in the same measure. If the measure consists

of multiple questions, they should all yield similar

answers. Discrepancies between items could indicate

that your measure is accidentally capturing two different

constructs simultaneously.

Likert scales

Many constructs, especially ones trying to capture beliefs, attitudes, and

experiences, are measured using rating scales called Likert scales (e.g., “On a

scale from 1 to 7, how often do you…”, where responses are ordered from

“Never” and “Rarely to “Often” and “All the time”).

Technically this is an ordinal measure, but it is typical to treat it as interval

in practice – for example, it is common to add up responses to create a sum

score or calculate a mean.

Missing data value

Missing data points

are given the NA

value.

Reliability

the consistency of our measurement

Test retest reliability

reliability: consistency across time (if we

repeat the measure at a later date, will we get the same

result?)

Interested reliability

consistency between raters (if

different people administer this measure, will they get

the same result?

Internal reliability

consistency across items (e.g.,

questions) in the same measure. If the measure consists

of multiple questions, they should all yield similar

answers. Discrepancies between items could indicate

that your measure is accidentally capturing two different

constructs simultaneously.

Validity

Validity is whether the measure is a good measure of the

underlying construct

Face validity

does the scale look like it measures

what it’s supposed to? Do others agree that it does?

Convergent validity

is the test related to other tests

that measure the same concept? I.e., do different

measures of the construct converge on similar results?

Discriminant validity

is the test capable of

distinguishing between the construct of interest and

an unrelated construct? E.g., a measure of reading

ability should not be measuring attention.

Predictive validity

is the measure related to later

outcomes?

Why are validity and reliability important

No measurement is perfect – we can estimate the degree of measurement

error in our assessment/experimental tools.

Measurement error influences the conclusions that can be drawn:

If a measure has poor reliability, we will be unable to observe real

relationships between the construct it aims to measure and another

construct.

This is because the relationship between two variables can’t be any stronger

than the relationship between either of the variables and itself (i.e., its

reliability)

Threats to validity of studies

Internal validity: integrity of findings

• Threatened by confounding variables: additional, often

unmeasured variables that are related to both the predictor and the

outcome

External validity: generalisability of findings

• Threatened by artefacts: factors that are unique to the special

situation in which you carry out your research; they could render

your findings applicable only to the artificial conditions of the study

(e.g., findings apply only to performance on artificial lab-based

tasks, not real-world behaviour)