Research Design exam 3

Two-group design

The simplest experiment

• 1 IV (2 levels)

• e.g. experimental and control conditions

• 1 DV

Inferential Statistics

Inferential statistics allow us to determine whether the alternative (experimental) hypothesis is true

• confirm or reject our prediction

dependent t-test

Compares two means on data from the same people measured at different times

• paired or matched samples

independent t-test

compares two means on data from different groups of people

rationale for t-test

We collect two samples of data and calculate and compare their sample means

Standard error gauges variability between sample means. if the difference between the samples is larger than what we would expect based on the standard error then we can assume one of the two

1) There is no effect, and the difference happened by chance

2) The two samples come from different populations (and so the null hypothesis is incorrect)

rational for the t-test

t= observed difference between sample means - expected difference between population means (if the null is true) divided by the estimate of the standard error of the difference between two sample means

there is more variability when the groups are

not the same

the two-group design is the simplest design that can yield a valid experiment

when you want to compare two means on data from the same people measured at different times, use a dependent samples t-test

multiple group design

Comparing several means

• 1 IV (3 or more levels)

• 1 DV

We can use a t-test to compare means, but this test has limitations

• We can compare only 2 means, but often we want to compare means from 3 or more groups

• It can be used only with one IV/predictor variable

ANOVA

• Compares several means

• can be used when we have manipulated more than one IV

• It is an extension of regression

What does ANOVA tell us?

• like a t-test, ANOVA tests the null hypothesis that the means are the same

• ANOVA is an omnibus test

• - it tests for an overall difference between groups'

• it tells us that the group means are different, but it doesn’t tell us exactly which means differ

H0

the means are the same

H1

The means differ

Theory of ANOVA

We calculate how much variability there is between scores

• total sum of squares (SSt)

We then calculate how much of this total variability can be explained by the model we fit to the data

How much variability is due to the experimental manipulation

model sum of squares (SSm)

How much variability is due to individual differences in performance

Residual sum of squares (SSr)

F- ratio

We compare the amount of variability explained by the Model (experiment) to the error in the model (individual differences)

What is the equation for F statistic

Between - groups of variances

Within - groups variance

SST Total variance in the data splits into what two groups

SSm: variance explained by the model

SSr: Unexplained variance

If the experiment is successful

the model will explain more variance than it cant explain

The final goal for the ANOVA is an F-ratio

F= Variance between treatments

. Variance within treatments

Each variance in the F-ratio is computed as SS/df

Variance between treatments = SS between

. df between

Variance within treatments = SS within

. df within

to obtain each of the SS and df values

the total variability is analyzed into the two components

Degrees of freedom (df)

the number of values that are free to vary
– In general, the df are one less than the number of values used to calculate the SS

step 2

calculate SSm

How many values did we use to calculate SSM?

We used the 3 means

step 3

calculate SSr

How many values did we use to calculate SSR?

We used the 5 scores for each of the SS for each group

step 4

Calculate the mean squared error

step 5

Calculate the F-ratio

step 6

Construct a summary table

Why use follow-up tests?

The F-ratio tells us only that the experiment was successful

• Example: the group means were different

it does not tell us specifically which group means differ from which

• we need additional tests to find out where the group differences lie

multiple t-tests

THIS IS A BAD IDEA

planned contrasts

hypothesis-driven, planned a priori

post hoc tests

not planned (no hypothesis), compare all pairs of means

planned contrast rules

• Variables not included in the contrast are coded as zero

• Positive values are compared to negative values

• entire row must sum to zero

post hoc tests

• Compare each mean against all others

• In general terms, they use a stricter criterion to accept an effect as significant

• - hence, control the Type I error rate