Validity

What is validity?

1. Extent to which an observed effect is genuine—does it measure what it’s supposed to measure? (internal validity), and can it be generalised beyond the research setting within which it was found? (external validity)

2. Observed effect = psychological test, observation, experiment

What is internal validity?

Whether the effects observed in an experiment are due to the manipulation of the independent variable and not some other factor

What is one threat to internal validity?

1. If p’s respond to demand characteristics and act in a way that they think is expected

2. E.g., P’s in Milgram’s experiment were criticised for ‘playing along’ with the experimental situation, not believing they were actually administering shocks

What is external validity?

1. Relates more to factors outside of the investigation

2. Generalising to other settings, other populations of people and other areas

What is ecological validity?

1. Type of external validity

2. The extent to which findings from a research study can be generalised to other settings and situations

What causes low ecological validity?

1. If the task that it used to measure the dependent variable in an experiment is not ‘like everyday ’life’—mundane realism

2. E.g., a researcher might give people a list of words to remember to assess memory and could do this in a shopping mall—field study

3. Setting doesn’t make the findings ‘more realistic’; using a word list makes the findings from this study lack ecological validity

What does this teach us?

We must look at all sorts of aspects of the research set-up in order to decide whether findings can be generalised beyond the particular research setting

What is temporal validity?

1. Type of external validity

2. The extent to which findings from a research study can be generalised to other historical times and eras

What are some examples of (low) temporal validity within psychology?

1. High rates of conformity within the original Asch experiments were a product of a particularly conformist era in recent American history (1950s)

2. Some of Freud’s concepts such as the idea that females experience penis envy, are deemed to be outdated, sexist and a reflection of the patriarchal Victorian society in which he lived

What is face validity?

1. A basic form of validity in which a measure is scrutinised to determine whether it appears to measure what it is supposed to measure

2. E.g., does a test of anxiety look like it measures anxiety?

What is concurrent validity?

The extent to which a psychological measure relates to an existing similar measure

What are the ways of assessing validity?

1. Face validity—whether a test, scale or measure appears ‘on the face of it’ to measure what it is supposed to measure—determined by simply ‘eyeballing’ the measuring instrument or by passing it to an expert to check

2. Concurrent validity—demonstrated when the results obtained are very close to, or match, those obtained on another recognised and well-established test

What is an example of concurrent validity?

1. E.g., a new intelligence test may be administered to a group of p’s and the IQ scores they achieve may be compared with their performance on a well-established test

2. Close agreement between the two sets of data would indicate that the new test has high concurrent validity—and close agreement is indicated if the correlation between the two sets of scores exceeds +0.80.

How can we improve validity through experiments?

1. Control groups—researchers are better able to assess whether changes in the dependent variable were due to the effect of the independent variable

2. Standardised procedures - minimise the impact of participant reactivity/investigator effects on the validity of the outcome

3. Single/double-blind procedures—designed to achieve the same aim, single-blind: P’s unaware of aim to reduce DC’s; double-blind: a third-party conducts the investigation without knowing its main purpose, reduces DC’s/investigator effects, improves validity

How can we improve validity through questionnaires?

1. Lie scales often incorporated within the questions

2. Assesses the consistency of a respondent’s response and controls for the effects of social desirability bias

3. ALSO anonymous data

How can we improve validity through observations?

1. Minimal intervention by the researcher—high ecological validity (especially in covert observations)

2. Broad behavioural categories that are overlapping or ambiguous may have a negative impact on the validity of the data being collected

How can we improve validity through qualitative data?

1. Depth/detail of case studies/interviews better able to reflect a p’s reality

2. Researcher may still have to demonstrate the interpretive validity of their conclusions—the extent to which researcher’s interpretation of events matches that of their p’s

3. Demonstrated through the coherence of the researcher’s narrative/inclusion of direct quotes from p’s within the report

4. Triangulation—the use of a number of different sources as evidence (e.g., data compiled through interviews with friends/family, personal diaries, observations, etc.)