Lecture 17: between-subjects design

what’s the difference between between-subject design, between-group design and independent-measures experimental design?

nothing, it’s the same thing

explain what’s a between-subject design

• different group of participants is assigned to each condition

• each group receives a different experimental treatment (IV) and they are all compared

what’s the key element of between-subject design?

• we use separate groups of participants for different conditions

• we then compare the data (DV) to across groups to look for differences

the treatment is the [DV/IV] and the data is the [DV/IV]

• treatment = IV (what we control)

• data = DV (what we measure)

why is between-subjects design an independent measure?

because participants only experience one level of the IV, meaning that there is only one score per participant

true or false: all IVs can be measured in both between-subject or within-subject designs

false: some IVs can, but other can be measure only with a between-subjects design

• ex (between only): age, gender

• ex (within or between): teaching method, video condition

why do we prefer between-subject designs for kids?

• they have limited attention span

• between-subjects allows one condition per infant, which is perfect for their attention

• ex: having one kid go through one condition (between-sub) VS having one kid go through three conditions (within-sub)

define “systematic variance”

difference in the DV between groups (between the means of different treatment groups)

define “non-systematic variance”

• scores varying within groups

• individual difference occurring by chance

what’s the difference between systematic and non-systematic variance?

• systematic: difference of the scores/DV between groups

• non-systematic: difference of the scores/DV within groups, individual differences occurring by chance

define “non-systematic variance”

scores that vary within the group, individual differences that occurred by chance

[systematic/non-systematic] variance is an important source of error that must be minimized

non-systematic: it’s the one that varies within-group, individual differences that occur by chance

what’s another name for “treatment index”

F-ratio

define “experimental error”

chance factors that cause differences across any condition and that cannot be controlled

*works for between or within

true or false: you can completely eliminate experimental error

false: it’s non-systematic variance, there will always be some difference

if we treated the groups in a between-subject designs the exact same way, would you obtain the same scores? explain why

no: there will always be some experimental error that will cause some differences between the mean

what are the components/causes of systematic/between-subjects variance? (2)

• treatment effect

• experimental error: factors that cannot be controlled and will cause differences across conditions

→ systematic variance = treatment effects + experimental error 

[systematic/non-systematic] variance is any differences between subjects who are treated alike

non-systematic, within-group: everyone goes through the same conditions

[systematic/non-systematic] variance = experimental error 

non-systematic 

treatment index/F-ratio = x ÷ y or (z+a) ÷ b

F-ratio:

• between-groups variance (systematic) ÷ within-group (non-systematic) variance

• (treatment effects + experimental error) ÷ experimental error

true or false: the F-ratio is not sensitive to the absence/presence of treatment effect (effect of the IV)

false: it is

if your F-ratio is large, you [can/cannot] reject your null hypothesis. if your F-ratio is low, you [can/cannot] reject your null hypothesis

• large = reject H0: Fstat > Fcrit

• small = retain H0: Fstat < Fcrit

large between-groups/systematic variance is [good/bad], large within-group/non-systematic variance is [good/bad] because […] 

• large systematic: good

• large non-systematic: bad

explanation

• reminder: F-ratio = systematic ÷ non-systematic 

• when you have a large denominator (within-subjects variance), you will get a small number

• you want a large F-ratio in order to see the effect of your IV and reject the H0

how can you avoid getting small F-ratios? (2)

• have large a numerator

• have a small denominator 

*reminder: F-ratio = systematic ÷ non-systematic variance 

• if the between-groups/systematic variance > within-group/non-systematic variance, then the F-ratio is [near 0 and small/positive and large]

• if the between-groups/systematic variance < within-group/non-systematic variance, then the F-ratio is [near 0 and small/positive and large]

• between > within, F-ratio = positive and large (good)

• between < within, F-ratio = near 0 and small (not so good)

what does the circled bar represent?

• it’s the standard error: value of the variance within each condition

• high bar = a lot of non-systematic variance (no good)

• small bar = not a lot of non-systematic variance (good)

define “single-factor analysis of variance” (ANOVA)

• test to determine if there are statistically significant difference between the means of three (or more) independent groups 

• there is one independent variable with three conditions 

• ex: compare driving performance (IV) with cellphone, handfree phone or no phone (conditions)

why would you use a single-factor analysis of variance (ANOVA) instead of doing multiple tests? (ex: phone vs handfree, handfree vs none, phone vs none)

an ANOVA will provide you with stronger evidence for cause-and-effect relationship than a two-group design

how can you maximize between-group differences?

by comparing two distinct things (distinct enough so that you can notice some differences)

how can you minimize within-group differences? (3)

• standardize experiment procedure so that all participants are treated the same way 

minimize individual differences:

• hold extraneous variable constant or restrict the range 

• use a large sample size

what are the major sources of confounding variables in a between-groups design? (2)

• individual differences/assignment bias: process of assigning participants to different conditions produces groups with different characteristics

  • ex: age, sex, IQ, socioeconomic status

• environmental variables: uncontrolled characteristics in the environment that may differ across groups

  • ex: room, lighting, time of day, noise

define “assignment bias” 

process of assigning participants in conditions produces group with different characteristics 

why is assignment bias a threat to internal validity?

because it offers an alternative explanation to the changes seen in the DV: is it because of the IV or because of how we assigned our participants?

how can you avoid experimental bias or individual differences?

by controlling or keeping fixed individual difference variables in each condition (ex: have an age range)

define “environmental variables”

uncontrolled characteristics in the environment that may differ across your groups 

true or false: environmental variables are only extraneous variables

false: if they differ between groups, they go from extraneous to confound as they will also affect the DV 

how can you establish equivalent groups of participants? (3)

• create them equally

• treat them equally

• compose them of equivalent individuals

define “randomization”

participants are randomly assigned to groups to ensure that groups are as equal as possible before treatment 

why is randomization the most powerful technique to control the effects of pre-existing differences?

because it equalizes/spreads the difference across all conditions

what’s the difference between random sampling and randomization?

• random sampling: random selection from the population that will constitute your sample

• randomization: random assignment into groups after the random sampling

define “free random assignment”

ensuring that participants are assigned to groups only based on chance (everyone has an equal chance of being assigned to one of the conditions)

true or false: free random assignment is guaranteed to lead to equality/non-systematic variance

false: not sure if you’ll end up with perfect equality because it’s at random

why is free random assignment used even if it won’t lead to perfect equality between the groups?

• the groups won’t be perfectly matched, but the difference is negligible

• the important differences that we will see between the groups will be caused by treatment  

true or false: free random assignment also works on small samples

false

define “matching”

match participants on pre-existing differences that may be related to differences in the DV so that they are equivalent on critical variables)

what are the steps of matching? (4)

1. identify the variable(s) to be matched

2. measure and rank subjects on the variable for which control is desired

3. segregate subjects into matched pairs on that variable 

4. randomly assign pair-members to the conditions 

true or false: you can also match in groups and not on individuals only 

true: 

• ex: high > 110, medium = 90-110, low < 90

define “attrition”

participants leaving the study before it’s finished

when does attrition become a problem? (2)

• when participants in one group leave more than participants in other groups as the groups are no longer equivalent (differential attrition)

  • _{*if equal participants from each group leave equally, then it’s not a problem}

• threat to internal validity: is the difference between groups is due to treatment or differential attrition

what’s the difference between attrition and differential attrition?

• attrition: participants leaving the study before it’s completed

• differential attrition: participants from one group leave more than participants from other groups

what are the communication between groups that can threaten internal validity? (3)

• diffusion/spreading: treatment effects from one condition spread to another condition

• true effects of treatment may be masked by shared information

• resentful demoralization: perceived inequality that influence behaviour

define “resentful demoralization”

perceived inequality that can affect behaviour (ex: knowing that other condition receives money but not you)

what are the advantages of between-subjects design? (4)

• simple design: each score is independent from other scores

• clean and uncontaminated by other treatment factors (no carryover effects, practice effects)

• experiment takes less time for each participant

• causality can be established

why are between-subjects preferred over within-subject?

• within-subject has carryover effects as the same participant goes through many conditions

• the design will have many types of carryover effects (first, second, nth order) that will increase with the amount of condition going through

• carryover effects create more bias as the number of conditions increase

• ex: smaller effect from A to B than from A to C

• which means that it’s biased: your response changes because of the experience they got from previous conditions and not by the treatments

• you don’t need to worry about this with between-subjects!

what are the disadvantages of between-subjects designs? (4)

• requires many participants as one participant contributes to one score only

• can be difficult to recruit enough participants in special populations 

• individual differences and environmental differences can exist: groups must be equivalent before manipulation 

• generalization (external validity) can be hard if you’re always holding constant the extraneous variable 

what are the solutions to counter assignment bias, experimenter expectancy bias and subject-expectancy bias? (4)

• participants are blind/don’t know the condition they are in

• experimenters are blind/don’t know the condition the participants are in

• both the participants and the experimenters are blind

• data analyst is blind (instead of saying treatment, placebo, we say 1, 2)

when should you use or not use a between-subjects deisgn?

• should: avoid carryover or comparison effects

• should not: when you want similar anchoring across conditions (ex: asking someone how high is the Olympic stadium and asking someone else how high is a tree… how high compared to what?)

define “comparison effects” 

within-subjects designs raise participants’ perception of difference across conditions 

what should you try to avoid when doing a between-subjects design? (4*)

• carryover effects

• participant awareness

• changes in measurement properties/response over time

• *ecological validity (you want ecological validity, participants are usually not exposed to all levels of a variable)