reliability and validity

Define reliability

How consistent findings from an investigation are/ measuring device is

• Something is reliable if it produces consistent results every time it is used (if repeated)

What are the two ways of assessing reliability?

• test-retest reliability

• inter-observer reliability

Describe test-retest reliability

Same person, 2 separate occasions:

1. same test/questionnaire to the same people on different occasions

   • Is the test is reliable, then results should be same each time administered

2. needs to be sufficient time between test & retest so participant cannot recall answers but not so long that their attitudes/opinions change

3. the 2 score sets are correlated

   • If the correlation is significant and strong, then we can conclude that the test is reliable

For data to be reliable, it needs to be a correlational coefficient of at least +0.8

Describe inter-observer reliability

Same occasion, 2+ observers

1. Observer ship conduct observation in teams of at least two, a pilot study should be conducted beforehand in order to check that the observers are applying behavioural categories in the same way observer should watch the same event but record data independently

2. One research is interpretation of the events may differ from someone else’s which introduces bias and subjectivity into the data collection process

3. The data collected by the two observers is correlated to assess reliability

What are the four ways of improving reliability?

Experiments, questionnaires, interviews, observations

Describe the four ways of improving reliability

Experiments

Questionnaires

interviews

Observations

Define validity

The extent to which an effect is genuine and can be generalised beyond the research setting

Define face validity

A basic form of validity in which a measure is thought to be valid if it appears to measure what it is supposed to measure

Define concurrent validity

The extent to which a measure refers to a current, existing, similar measure

Define ecological validity

The extent to which findings from a research study can be generalised to other settings and situations

Define temporal validity

The extent to which findings from a research study can be generalised to other historical times and eras

What are the four ways of improving validity?

Qualitative methods, questionnaires, observations, experiments