Reliability

What is reliability

A measure of consistency - if a particular measurement is made twice and produces the same result then that measurement is described as being reliable

What is test-retest method

A way of assessing reliability. This is when the same test or questionnaire is given to the same person (or diff people) on different occasions. If the test or questionnaire is reliable than the results obtained should be the same, or very similar. There must be sufficient time between test and retest to ensure the participant can’t recall their answers, but not so long that their attitudes/opinions/abilities have changed. The two scores would then be correlated to make sure they are similar - if the correlation is significant and positive than the reliability of the measuring instrument is good

What is inter-observer reliability

The extent to which two or more observers agree. This may involve a pilot study of the observation to check observers are applying the behavioural categories in the same way, or a comparison may be reported at the end of a study. Observers watch the same event and record data independently. The data collected should be correlated.

What is reliability measured using and what should the correlation coefficient exceed for reliability

Measured using correlational analysis. Should exceed +.80 for reliability

How do you improve the reliability of questionnaires

Using the test-retest method - comparing two sets of data should produce a correlation that exceeds +.80. If a questionnaire produces low test-retest reliability some of the questions should be revised or rewritten (change to close-ended questions)

How to improve reliability of interviews

Use the same interviewer each time. If this is not possible then all interviewers should be properly trained

How to improve reliability in observations

Make sure behavioural categories have been properly operationalised. Categories should also not overlap and all possible behaviours should be covered on the checklist.

How to improve reliability of experiments

Procedures must be standardised