Unit 2 - Reliability

What is meant by ‘reliability’?

How consistent something is, and consistency refers to how stable, uniform and dependable something is.

What is ‘internal reliability’?

The extent in which a test or measure is consistent within itself.

What is ‘external reliability’?

The extent to which a test produces consistent results over several occasions.

What are the most common issues of internal reliability?

Poor operationalisation of dependent and independent variables, order effects, lack or standardisation and accuracy (or lack of) of measuring tool.

What can be done to deal with order effects?

Counterbalancing - this refers to researchers splitting the sample into two halves, and having the groups do the conditions in the opposite order, which eliminates order effects, increasing reliability.

What can be done to deal with lack of standardisation of procedures?

Standardisation - where researchers following a standard set of instructions when conducting research. Research should also be conducted in the same environment, and as many elements should be controlled as possible. This allows the researcher to establish cause and effect through reliable data collection.

What can be done to deal with accuracy (or lack of) of measuring tool?

Inter-rater reliability - increases consistencies when multiple researchers observe or collect data as it ensures that they are identifying the same problems/solutions.

What are issues of ‘external reliability’ related to?

Consistency of results over time and across different populations. The more controlled a set of procedures and measures are, the easier they are to replicate and the more likely you are to get similar results.

What are the three ways to assess reliability?

Split-half reliability testing, test-retest reliability and inter-rater reliability.

What does ‘split-half reliability testing’ entail?

Splitting the test answers from a participant in half and seeing whether the individual got the same or similar scores on the two halves. If so, internal reliability is high.

What does ‘test-retest reliability’ entail?

Testing and retesting the same participants over time with the same test and comparing their scores. If their scores are the same, they have high external reliability.

What does ‘inter-rater reliability’ entail?

Where two or more researchers produce consistent results by using a standardised procedure, an agreed coding system or a correlation of their data.