Research methods week 2

Ways of developing research

• Observation

• Serendipity(right place at right time)

• Everyday problems in need of a solution

• Replication and Extension

• Developing research from others research

• 
  

What is a P/Ps

Participant(s)

What is an IV

Independent variable (cause)

• This is a variable which the experimenter systematically varies or manipulates

• Should have at least 2 levels (for comparison)

What is a DV

Dependent variable (effect)

• This is an outcome or measurement variable and is the variable upon which the experimenter is interested in observing effects

Cause X → Effect Y. What is X and Y

X- Independent Variable

Y- Dependent Variable

What are the three types of Independent Variables

• Situational

-E.g., Number of bystanders in a helping behaviour study

• Task Variables

- E.g., Divided into groups, then given different logical problems to solve and measuring performance

• Instructional Variables

- E.g., Divided into groups, then given instruction to memorise objects by using visual images OR no instruction at all and then measuring task performance

 

What is the control group/condition

The absence of a manipulation

What is the Experimental group/condition

 Manipulation is administered

Participants are placed into experimental and control groups by a process of ? and why?

Random Assignment -

• This rules out the possibility that there are systematic differences (e.g., in intelligence, personality, belief in luck) between the groups. 

What does Randomisation involve

 Strictly non-systematic assignment of Ps to conditions

Often it isn’t possible to manipulate the IV directly, so what is done instead?

Psychologists typically manipulate theoretical variables indirectly, and then check that the IV was manipulated successfully.

This is achieved by using a ‘manipulation check’

What is an indirect manipulation

Manipulating an IV indirectly by creating conditions designed to change a psychological state.

Why do we perform a manipulation check

To be sure that this manipulation has had the desired effect on the participants’ attributions

–The purpose of this is to ensure that the manipulation has had the desired effect.

Manipulation check example

• For example, after the above questions the researcher may ask,  ‘To what extent were external factors responsible for your failure’, asking participants to respond on a scale with end-points labeled ‘not at all’ and ‘completely’. 

• If participants who are asked ‘internal’ questions consistently select lower numbers than participants who are asked ‘external’ questions, then we conclude that the manipulation has been successful.

• In this way, we can check that our manipulation has achieved what we expect

• We can then go on to test for differences on our DV, knowing if our ‘internal’ group are actually making more internal attributions for failure and our ‘external’ group are making more external attributions

What is an extraneous variable

• A variable that is not of specific interest to researcher, but might influence the behaviour under investigation (i.e., the DV)

Any variable that is not of immediate interest to a researcher, but which may pose a threat to validity because it compromises the interpretation of research findings. This is usually because it obscures the measurement of the processes of interest.

It generates ‘noise’ in our data

What can happen if extraneous variables are not carefully controlled and what does this lead to

• Some extraneous variables may systematically influence the DV

• The result is ‘confounding’

What is a Confounding Variable

•This is a situation where extraneous variable co-varies with the IV and could provide alternative interpretation of the results.

•A systematic effect of an extraneous variable on a DV could be mistaken for the effect of the IV.

Unintended or accidental manipulation of an extraneous variable that occurs because that variable (a ‘confound’) is associated with an independent variable (IV) in an experiment.

Its effect on the DV might be mistaken for the effect of the IV

Identify DV and IV example.

DV- Performance in the exam (%)

IV- Time spent studying the text (hours)

What’d important to keep in mind with EV and CVs

•Not every experimental flaw or extraneous variable is a confound!

For example, imagine that this study only recruited participants over 60 years-old

Age may NOT be a confound here:

•We are comparing exam % between groups 1, 2 and 3

•As long as age of Ps in each group is similar, that’s okay

–But it may limit external validity of the study:

We cannot generalise our findings beyond the age-range recruited for the experiment

•For example, if Ps are all 60yrs+, we can’t claim that the findings can be generalized to another group (e.g., school-age children)

They might be, but we have no evidence for the claim

How to measure the DV?

–Refer to previous research

–Run a pilot study

What is a pilot study

A small preliminary study used to test and improve the procedure or measures before the main study.

What can pilot studies help identify

Ceiling effect-If task is too easy, all scores will be very high. Differences between Ps will be disguised.

Floor effect-If task too difficult, all scores very low. Differences between Ps will be disguised.

What is a solution to the ceiling and floor effect

•Identify task of moderate difficulty, determined through pilot testing.

Why can selecting a dependent variable (DV) be difficult?

Practical, ethical, and legal constraints may prevent researchers from measuring the real-world outcome directly.

What two qualities should a good DV have?

It should be relevant to the outcome of interest and sensitive enough to detect changes caused by the IV.

What trade-off can occur when selecting a DV?

A safer and more practical measure may be less realistic, while a realistic measure may be unethical or impractical.

When choosing a DV, the researcher must:

balance relevance, sensitivity, practicality, ethics, and ecological validity

What is the relevance–sensitivity trade-off when choosing a DV?

•The more sensitive a DV is to changes in the IV, the less relevant it may become to the real-world phenomena in which one is interested. 

“The principle that the more relevant a dependent variable is to the issue in which a researcher is interested, the less sensitive it may be to variation in the independent variable.”

“The more sensitive a DV is to changes in the IV, the less relevant it may become to the real-world phenomena in which one is interested.”

Haslam & McGarty (2019)

In some areas of research, the link between the DV and the ‘real world’ becomes extremely tenuous, and this can seriously undermine the experiment’s what?

External validity

How can the relevance–sensitivity trade-off affect external validity?

If the DV is too removed from the real-world phenomenon, the findings may not generalise well outside the experiment.

What is a proxy measure?

An indirect measure used when the actual variable of interest cannot be measured directly.

Why might researchers use a proxy measure as the DV?

The real outcome may be unethical, impractical, or difficult to measure directly.

What is a problem with proxy measures?

They may not closely enough represent the variable they are intended to measure.

Relevance sensitivity trade off example

1.DV ‘running down pedestrians’ is most relevant (it is actually what we want to explain or predict), but the relationship may not be particularly strong (sensitivity of changes in DV in response to manipulation of IV may be low) and so difficult to detect. HIGH RELEVANCE, LOW SENSITIVITY

2.DV ‘general impairment’ is slightly less relevant to what we are interested in (running down pedestrians) as it doesn’t take into account other variables such as driver skill or experience, but changes in DV in response to manipulation of IV will be easier to detect (sensitivity is higher). LOWER RELEVANCE, HIGHER SENSITIVITY

3.DV ‘reaction time’ is even less relevant to running down pedestrians, but changes in DV in response to manipulation of IV will be easiest to detect. LOWEST RELEVANCE, HIGHEST SENSITIVITY

The lower the relevance of the DV to the thing we are actually interested in, the easier the IV/DV effect may be to detect and measure.

What are quasi-experimental variables

–Variables that cannot be manipulated, but we can select people for each condition based on their characteristics.

–They are self-selected

–Introduces the need for additional considerations in order to avoid possible confounds, depending on the nature of the variables

What is an issue with quasi experiments

Self selection bias

What is self selection bias

Bias caused when volunteers differ systematically from non-volunteers, making the sample less representative.

It mainly threatens external validity/generalisation, because the findings may not apply as well to people who wouldn't choose to participate.

Example of self selection bias

–Allied aircraft in the Second World War (1939-45)

• Bullet damage sustained by Allied aircraft returning from bombing missions

• At first the military wanted to reinforce those areas that appeared most susceptible to bullet damage

• Until a mathematician, Abraham Wald, pointed out that these were the aircraft that had made it home and so were self-selected

• It was all the other areas that should be reinforced

Experimental vs quasi experimental studies

•Experimental studies

–Extraneous variables are controlled for

–Causal inference can be made

–We can conclude that changes in IV cause changes in DV

Quasi-experimental studies

—Extraneous variables can’t be controlled

—Causal inference can’t be established

—We cannot say that the IV is a cause of the DV

—We can only say that the groups (IV) performed differently on the DV.

Example- A study on the effect of confidence (‘self-efficacy’) on risk-taking behaviour

–Option 1: Manipulate confidence experimentally (e.g., using feedback on performance), random assignment to high and low confidence conditions, then measure risk-taking behaviour

–Option 2: Measure participants’ confidence, and on the basis of their scores separate them into groups with high and low confidence, then measure risk-taking behaviour

Which is better?

Option 1: Offers opportunity to establish causality and so allows us to develop an explanation for the IVÞDV relationship,

Thus, results from Option 1 can contribute to relevant theory (see last week’s lectures - ‘a good theory involves explanation, not just description’)

Option 2: We cannot infer causality, we can only describe what happened, and so the possible contribution to theory is much more limited