Topic Three: Threats to Internal Validity

What is participant reactivity?

when participants react to cues, often unintentionally, in an experimental situation

What is the effect of participant reactivity?

can affect the validity of conclusions drawn from the collected data

What are some examples of participant reactivity?

Social desirability bias and the Hawthorne effect

What is social desirability bias?

when participants want to be seen in a good light, leading them to lie or behave in a different way, or not give true opinions, thinking that it will make others think good of them

What is the Hawthorne effect?

the attention gained from being in a study impacts the behaviour of the participants, impacting the validity of the conclusions drawn

Outline the findings of Roethlisberger and Dickinson (1939)

production levels continued to increase even when lighting levels were changed in a factory, they then realised that the increased attention of the workers was impacting the findings of the study.

What are investigator effects?

when subconscious clues from an investigator encourages certain behaviours from the participants

What can investigator effects include?

anything that the investigator does that affects the performance of a participant, other than what is intended

What can investigator effects lead to?

participants’ fulfilment of the investigator’s expectation

Outline Greenspoon (1955) and the Greenspoon effect

participants’ behaviour could be altered by saying ‘mm-hmm’ after certain responses (e.g. after a desired behaviour to encourage the participant to repeat that behaviour)

How can validity be assessed?

Face validity and concurrent validity

What is face validity?

Does it look like conditions are measuring what they intend to?

What is concurrent validity?

Comparing new test with an already established test that measures the same characteristic.

Correlation of 0.80 or higher desired

How can validity be improved?

Standardised procedures

Ensure participants knows that self-report data is anonymous

Use lie scale/filler question for questionaires

Use covert observations

How can standardised procedures improve validity?

Standardised procedures and instructions limit extraneous variables

Use of single and double-blind procedures for experiments

How can reliability be assessed?

Split-half method

Test-retest method

Inter-observer reliability

What is the split-half method?

Split test in half and score individually

Want correlation of 0.80 or higher between each half

What is the test-retest method?

Test participants once, repeat the same test and see if there’s consistency

Want correlation of 0.80 or higher between the two tests

What is inter-observer reliability?

Have multiple observers rate the same behaviours and see if there’s consistency (+0.80)

How can reliability be improved?

Avoid complex or ambiguous questions in questionnaires

Use same interviewer or ensure all are fully trained - structured interviews

Standardise procedures and instructions for experiments

Behavioural categories should be clear, measurable, not overlap, and all behaviours should be listed