Reliability and Validity

Define reliability

• whether something is consistent

• Researchers should be able to replicate a method time and time again

• and get the same or similar results

• Reliability is important in research because we can be more confident of the conclusions

What are the two ways of measuring relaibility?

• Test-retest method

• Inter-observer reliability

Explain the Test-retest method

• same participants take the same test twice, at different occasions

• If the results from the two occasions highly correlate (with a correlation co-efficient of at least +0.8)

• the test is said to be reliable

• This method is most commonly used with questionnaires and psychological tests (e.g. IQ tests)

• There must be sufficient time between test and retest to ensure the participant cannot recall their answers

• but not so long where the participant has significantly changed

Explain Inter-observer Reliability

Inter-observer reliability is used to measure the reliability of observations. Two (or more) observers watch the same behaviours with the same behavioural categories. The observers watch the same events, but record their behaviour independently. If the results from the different observers highly correlate (with a correlation co-efficient of at least +0.8), the observation team is recording behaviour reliability. A similar process can be used to assess the reliability of interviews (inter-interviewer reliability) and content analysis (inter-rater reliability).

Define Validity

Validity refers to whether a method measures what it claims to be measuring. Validity is essential in order to give confidence in the findings and conclusions.

Define Internal Validity

Internal validity is the extent to which findings obtained are a direct result of the variable being manipulated. In an experiment, internal validity refers to whether changes in the DV are caused by manipulation of the IV rather than extraneous variables.

What are the measurements of internal validity?

• Face Validity

• Concurrent Validity

Explain Face Validity

Face validity only requires intuitive measurement of whether a test appears to be measuring what the researcher intends to measure. If a test produces expected results, it is said to be valid. For example, if questions on a stress questionnaire are all related to stress, or a person who feels stressed obtains a high score, the questionnaire has face validity.

Explain Concurrent Validity

Concurrent validity involves measuring if a test used in a current study produces a similar measure of a variable when compared to a previously validated test. Participants are given both measures at the same time and the results are compared. If the results from the two tests highly correlate (with a correlation co-efficient of at least +0.8), the test in the current study is said to have high concurrent validity.

Define External Validity

External validity is the extent to which findings can be generalised beyond the investigation to settings, different periods of time, and other groups of people (population validity).

What are the measurements of external validity?

• Ecological Validity

• Temporal Validity

Explain Ecological Validity

Ecological validity is a type of external validity and refers to whether the findings can be generalised from a study to other settings, particularly to real life situations. It can be affected by the environment the study is carried out in (controlled/natural), how the DV was measured and whether the participants know they are being assessed. If the task used to measure the DV is not similar to a task we carry out in everyday life we may say the study has low mundane realism and this can lower the ecological validity.

Explain Temporal Validity

Temporal validity is a type of external validity and is the extent to which findings from a research study can be generalised to other historical times and eras. Critics have suggested that high rates of conformity within the original Asch experiments were a product of a particularly conformist era in recent American history (1950s).