Year 2: Validity and Reliability

0.0(0)
Studied by 1 person
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/23

flashcard set

Earn XP

24 Terms

1
New cards

What is reliability?

The consistency of a measure - if it produces similar results each time.

2
New cards

What are the two types of reliability?

  • Test-retest reliability

  • Inter-observer/rater reliability


3
New cards

What is test-retest reliability?

The same test or interview is given to the same participants on two occasions to see if the same results are obtained.

  • There must be sufficient time between the test and the retest


4
New cards

What should the correlational coefficient be to show good high reliability?

0.8 or above

5
New cards

How do you check test-retest reliability?

  1. The researcher gives a test/questionnaire to a group of participants.

  2. The SAME participants complete the SAME test again on a different occasion (e.g. one month later).

  3. Each participant's first and second scores are correlated and can be plotted on a scattergram.

  4. The strength and direction of the correlation are assessed using an appropriate statistical test (Spearman's Rho/Pearson's r).

  5. The correlation coefficient is compared against a statistical table to determine whether the correlation is statistically significant.

  6. A strong positive correlation (+0.80 or above) suggests good test-retest reliability, meaning the test produces consistent results over time.


6
New cards

What is inter-rater reliability?

The extent to which there is agreement between two or more observers involved in observations of a behaviour.

7
New cards

What is the correlation ‘test’?

  1. Two sets of data are compared to identify whether there is a relationship between them.

  1. Spearman's Rho or Pearson's r is used to calculate a correlation coefficient.

  2. The coefficient ranges from -1 to +1, indicating the direction and strength of the relationship (+0.80 or above indicates good inter-rater reliability).

  3. Statistical tables are used to determine whether the correlation is statistically significant.


8
New cards

Improve reliability in observations

  • Operationalise all behavioural categories so that they’re measurable and self-evident

  • Ensure that the categories do not overlap (e.g. hugging and cuddling), as different observers have to make their own judgement of what to record, leading to inconsistent records


9
New cards

Improve reliability in questionnaires

  • Low test-rest may be due to poorly written questions, researchers need to reduce ambiguity

  • Removing open questions (can cause misinterpretation) and fixed choice questions (less ambiguous) may increase reliability


10
New cards

Improve reliability in interviews

  • Ensure that the interview is structured

  • Ensure the same interviewer is used every time

  • Interviewer should be trained not to ask leading questions


11
New cards

Improve reliability in experiments

  • Ensuring procedures are standardised

  • So that performance of different participants can be easily compared

  • Use a laboratory experiment - more control over extraneous variable


12
New cards

What is validity?

The ability of a test to measure what it is intended to measure, i.e. its 'truthfulness', if it can be generalised beyond the research setting within which it was found.

13
New cards

What are the types of validity?

  • Internal

  • External

    • Ecological

    • Temporal

  • Face validity

  • Concurrent validity


14
New cards

What is internal validity?

Whether the effects observed in an experiment are due to manipulation of the independent variable and not some other factor.

  • A threat to internal validity is that participants may respond to demand characteristics.


15
New cards

What is external validity?

A measure of whether data can be generalised to other situations outside the research environment, such as other settings, other populations of people and other eras.

16
New cards

What is ecological validity?

The extent to which findings from a research study can be generalised to other settings and situations - predominately real life.

17
New cards

What is temportal validity?

The extent to which findings from a research study can be generalised to other historical times/eras.

18
New cards

What is face validity?

The extent to which test items look like what the test claims to measure. Conducted by a specialist in the given area, if they believe the instrument is valid, this is a good indicator of validity.

19
New cards

What is concurrent validity?

Establishing validity by comparing it to an existing test or questionnaire with the one you are interested in. If the results are very close/match those obtained on another recognised and well-established test, that can be an indicator of validity.

20
New cards

What is mundane realism?

Whether the task used to measure the dependent variable represents real world events/use.

21
New cards

Improve validity in observations

  • Covert observations can increase validity, as the behaviour is more likely to be natural and authentic

  • Ensure that behavioural categories do not overlap, are not ambiguous and are clearly defined (operationalised)


22
New cards

Improve validity in questionaires

  • Incorporate lie scales (to identify participants who may be dishonest) to eliminate demand characteristics such as social desirability bias

  • Tell participants that results will remain anonymous so participants are less likely to give socially desirable answers


23
New cards

Improve validity in experiment

  • Ensure control groups are present to assess whether the changes of the DV is definitely due to the effects of the IV

  • Standardised procedures to minimise participant reactivity and investigator bias (double and single blind procedures)


24
New cards

Improve validity in qualitative methodology

  • Direct quotes - demonstrate interpretative reality (the extent to which the researcher's interpretation of events matches those of their participants)

  • Triangulation - using various different resources for evidence (interviews with family members, diaries and observations.)