Reliability & validity

Reliability

• The measure of consistency of a study

• A study is reliable if it is repeated using standardised procedures & the same results are obtained 

What are the ways of assessing reliability?

• Test-retest method - giving the same test to the same participants on a different occasion to see if the same results are obtained

• Inter-observer reliability - the level of agreement between 2 or more researchers/observers. Reliable if there is a positive correlation of +0.80

How can you improve reliability in a questionnaire?

• Use standardised closed Qs - replicable & easy comparisons between answers as quantifiable

• Avoid misleading/vague Qs

How can you improve reliability in an interview?

• Use same interviewers

• Closed Qs - quantifiable answers

• Structured interview - replicable

How can you improve reliability in an experiment?

• Control extraneous variables

• Standardised procedures

How can you ensure an observation is reliable?

• 1. Conduct a pilot study

• 2. Establish behavioural categories (operationalised)

• 3. Decide on time or event sampling

• 4. Conduct observation with 2 or more observers (if you can’t get 2 film if ethical to do so)

• 5. Compare findings

• 6. Must be 80% agreement to establish inter-observer reliability

Validity

The extent to which studies are accurate & actually measures what it claims to be measuring 

Face validity 

The extent to which a test is measuring what it claims to be measuring

Concurrent validity

• Comparing a new test with an existing test of the same nature to see if they produce similar results 

• A strong positive correlation of +0.80 indicates validity

Temporal validity

The extent to which findings are valid over time (external validity)

Ecological validity

• The extent to which behaviours & findings of a study are reflective of real life behaviour

• If ecologically valid then generalisable to other settings