Final exam 305

What is the main question we want to answer with repeated measures ANOVA?

Are there differences in the mean scores of the dependent variable across groups/ conditions?

What is another name for repeated measures?

within-subject designs

What is a within-subjects design?

• subjects are crossed with at least one experimental factor

• a.k.a before and after treatment design (2 conditions)

• each participant experiences each level of a factor/ each condition

what is a single-factor within-subject design

• a one-way repeated measures design

• n subjects are measured on the DV under k conditions (or levels) of a single IV or factor

  • ex. 8 children whose cortisol levels were measured 4 times in each child (pre, after 20 mins, after 40 mins, after 60 mins)

What are some possible research questions?

• 1. are there differences in the mean scores of the DV across groups/conditions?

  • within-subject effect of the IV (each subject is measured at each time point)

  • can be used to examine treatment effect of the IV (H0: μ1= μ2…=μk)

• 2. are there differences across subjects?

  • the variability of subjects (between- subject effect)

  • used to test the effect of the subjects (variance between subjects→ H0: Vs=0→ ex. all infants have the same level of cortisol)

One-way repeated measures ANOVA interests:

• usually we are NOT interested in the effect of ‘subjects’ or subject-level variability (difference from between subjects)

• if an effect exists, it would simply tell us that the subjects differ, but this has nothing to do with our treatment (IV) so it is irrelevant

• we are interested in whether the IV has an effect on the subjects, regardless of whether differences existed naturally among subjects (factor A)

what is SS(T)?

Total sum of squares

what is SS(A)?

Variation between group means (IV)

What is SS(S)?

Variation between row means (=subject means)

what is SS (AxS)?

Variation between cell means (values within the cells)

Assumptions in one-way repeated measures ANOVA:

• Normality: the distribution of observations on the dependent variable is normal within each level of the factor

• Homogeneity of variance: the population variance observations is equal at each level of the factor (diagonal terms in a table are equal)

• Homogeneity of covariance: the population covariance between any pair of repeated measurements is equal (homogeneous covariance)→ all off-diagonal terms are equal to eachother

  • ex. difference in variance between levels are the same →covariance (A,B)= covariance (A,C)= cov (B,C)

**Blue= compound symmetry = when both assumptions are met

Compound symmetry can be replaced with which assumption?

• can be replaced by the assumption of sphericity: differences between levels of the factor, have the same variance of the differences for each level

• In the assumption of sphericity, we assume that the relationship between pairs of experimental conditions is similar

• Assumption of sphericity is necessary for validity of the F test in repeated measures ANOVA

What happens when sphericity is violated?

• Tests: Mauchly’s W (1940)

  • H0: variances of differences between conditions are equal. (sphericity is met)

  • If p<0.05 (reject null=significant effect), the assumption of sphericity (and Compound symmetry) is violated

  • available in JASP

• When compound symmetry is violated, the F-tests in one-way repeated measures ANOVA tend to be inflated, leading to more false rejections of H0

  • these violations require adjustments to the F test

Approaches to deal with violation of sphericity:

• use a conservative critical value based on the possible violation of sphericity (conservative Ftest)

  • inflation of Ftest that occurs when sphericity is violated can be adjusted by evaluating the observed F value against a greater critical value, obtained by reducing the degrees of freedom.

  • DF (A)= Ɛ(k-1)→for factor in design→ with adjustment

  • DF(AxS)= Ɛ(k-1)(n-1) → for residual in design→ no adjustments=sphericity met then these are our DF for fcrit and f-observed

What are the statistical notations for when sphericity is met or violated?

• sphericity met: Ɛ=1 (i.e., no correction needed)

• sphericity violated: Ɛ<1 but can be anywhere from 0-1

  • when multiplied by epsilon, reduces DF (A) and DF (AxS), and gives a larger critical value for F-critical

How do we decide the value of Ɛ?

• Ɛ > or equal to 1/ K-1 (lower bound=strictest correction or most extreme)

• JASP provides two estimates of Ɛ

  • Greenhouse-Geisser (G-G) & Huynh-Feldt (H-F) estimates (correction epsilons)

  • G-G is smaller or more conservative → lower power to detect an effect→ corrects df more (stricter)

  • H-Y is closer to 1, higher power than G-G

  • both G-G and H-Y have higher power than using most extreme correction

what is compound symmetry?

the joint assumption of homogeneity of variance and homogeneity of covariance

what does Ɛ measure?

Ɛ measures the extent to which sphericity was violated (not computed manually)

what is the major difference between within-subjects and between-subjects ANOVA?

• within-subjects is not concerned with SSs or the effects of subject variability

• within-subjects is concerned with factor A

• within subjects only has 1 f-ratio

what is the null hypothesis for the model effect of IV?

H0: μ1=μ2=…μk (depending on number of levels) all means of all the levels of factors are the same

• enters computation of variation due to the model (SSA)

• can also calculate effect of subjects (between participant effect) and treat each as a different level in an experimental design→ SSs but not relevant

what are the steps of the APA summary in repeated measures ANOVA?

1. summary (levels of factor, type of ANOVA)

2. Was sphericity met or not using Mauchly’s test? → state whether met and the exact p-value

   • state which measure of correction was used if sphericity not met→ H-F or G-G

3. The results (significant or not), state IV and DV, appropriate F statistic depending on the test used, P-value used, and partial w²

4. always mention what type of follow-up is needed if there are significant effects

what does SS(T) partition into in one-way reapeated measures?

SSs and SSw but we are only interested in SSw

what does SSw partition into?

SS(A) and SS(AxS)

what does SSA partition into?

MSA to then calculate the F ration (Fa)

What does MS(AxS) partition into?

FS and FA but we are only interested in the F-ratio for factor A

What does a Two-way ANOVA mean

• comparing more than 2 means with two factors

• each factor has two or more levels

• multiple null-hypotheses concerning

  • main effect of Factor A

  • main effect of Factor B

  • interaction between Factor A and B

two way factorial experiments are…

• experiments with TWO independent variables or factors

• fully crossed: contain all possible combinations of the levels of factors (ex. 3×3, 2×2)

• experiments that contain info about: two main effects + interaction effect

what is assumed about two-way factorial experiments

• subjects serve in only one of the treatment conditions (independent sample or between-subjects designs)

• if samples are equal in each condition it is called a balanced design

what are main effects?

• the effect of one factor when the other factor is ignored (ex. main effect of factor A is a comparison to factor B)

• the differences among marginal means for a factor(A and B)

what is an interaction effect?

The extent to which the effect of one factor depends on the level of the other factor:

• present when the effects of one factor on the DV change at different levels of the other factor

• indicates that the main effects of the factors do not fully describe the outcome of the factorial experiment

• sometimes called crossover effect

• considers pattern results for all cell means

What is the purpose of Two-way ANOVA

• statistically examines the effects of:

  • two main factors of interest on the DV (main effects— row and column)

  • interaction between the different levels of these two factors (interaction effect)

what are the prior requirements/ assumptions for two-way ANOVA?

• DV is normal within each group

• variance of the population is equal for each group (homogeneity of variance)

• independence of observations

Hypotheses for main effects:

• main effect of factor A:

  • HoA: μA1= μA2…= μAa (equal row marginal means)

  • H1A: Not all μAg are the same (not all marginal means of factor A are equal to each other)

• main effect of Factor B:

  • HoB: μB1= μB2…= μBb (equal column marginal means)

  • H1B: Not all μBj are the same (not all marginal means of factor B are equal to each other)

hypothesis for interaction effect:

• H0, AXB: all μAgBj are the same OR the interaction between Factor A and Factor B is equal to zero

• H1, AXB: not all μAgBj are the same OR the interaction between Factor A and Factor B is NOT zero

Basic concepts of two-way ANOVA

1. divides the variance observed into different parts resulting from different sources

2. assesses the relative magnitude of the different parts of the variance

3. examines whether a particular part of the variance is greater than expectation under the null hypothesis

What does SSM partition into in two-way ANOVA?

• SSA: variation between means for Factor A

• SSB: variation between means for Factor B

• SSAxB: variation between cell means (interaction)

How to evaluate the F-ratio?

• same way as usual

• if the f-observed is greater than f-critical, we reject the null hypothesis

• or look at p-value—If greater than or equal to 0.05, reject the null

calculating effect sizes

Use omega squared for each main effect and the interaction effect

what is the appropriate follow-up when we find a significant effect for the interaction (AxB)?

simple main effects because we can’t say anything about the main effects without knowing which factors we’re looking at

what is the follow-up when we find significant results for our main effects?

post-hoc tests

what does SS(T) partition into in two-way ANOVA?

SS(M) and SS(R)

How many effects are there in a three-way ANOVA?

7 effects:

• 3 main effects (A,B, C)

• 3 simple (two-way) interactions

• 1 three-way interaction (ABC)

What is the follow up when we have a significant two-way interaction

simple main effects

what is the follow up when we have a significant three-way interaction

• Two-way ANOVAs→ pick one of the factors to do the two-way ANOVA and the remaining two are the factors for that ANOVA

what is the order in which we report significant effects in APA summary?

• start with three-way if significant, then two-way and main effects last

what is the appropriate follow-up when factor B is significant

post hoc tests

what is the appropriate follow-up when we have a significant effect of factor AxB?

simple main effects

what is the appropriate follow-up when we have a significant three-way interaction?

• two way ANOVA for 2 of the 3 factors at the level of the third factor

• the two way interactions vary for each level of the third factor (ex. A and B vary based on level of C)

definitional formulas

represents the meaning of the parameter

ex. population variance states that it is the mean squared difference between a score and the mean of all scores

computational formulas

equations used to calculate values for a statistical concept

• the definition of the construct may not be immediately obvious from the computational formula but it is algebraically equivalent (produces the same numerical value) to the definitional (conceptual) formula

• generally used to speed up calculations

notation for computational formulas (one-way)

xij: the raw scores on the DV

i: individual and j: group

Treatment sums: A and Aj

Grand Sum: T

what is statistics good for?

answering research questions

what is a sample?

A subset of a population. Usually research focuses on populations

N= sample size

Difference between descriptive and inferential statistics

• descriptive: summarize/describe properties of the sample (or the population)—>how are data distributed: central tendency, variability, shape of dist.

• inferential (this class): draw conclusions regarding properties of the population, but based on sample data (i.e. comparing sample mean to population mean)

What 3 things do statistical analyses depend on?

1. study design/research

2. types of variables (levels of measurement, distribution)

3. whether assumptions of the analyses are met

Independent vs dependent variables

• IV: predictor/covariate, factors in an experimental design

• DV: outcome/response, predicted variables

correlational vs experimental research

• correlational:

  • IV is measured by the researcher

  • no manipulation/ naturalistic

  • good for ecological validity

  • not good for inferring causality (i.e. confounding variable)

• experimental:

  • IV is manipulated by the researcher

  • good for inferring causality

  • manipulating IV in lab settings may feel detached from real world

  • may have similar statistical methods to analyze data

  • not good for external validity

what is the first type of research?

1. between-subjects design: each participant is only in one experimental condition (i.e. control or treatment)

   • Groups should be equal on any confounding variables if random assignment.

   • allows generalization between population and sample

   • allows us to infer causality of the treatment

what is the second type of research?

2. within-subjects design: each participant does more than one experimental condition (i.e. control and treatment)

   • DV measured multiple times

     • limitation: vulnerable to practice effects (# of times repeated measure has been done) and fatigue/bordom effects and alternative explanations for differences between conditions

     • counterbalancing to rule out these alternative explanations of repeated measures designs

what are the 4 levels of measurments?

1. nominal: classifies objects (dichotomous=binary)

2. ordinal: categories with ranking

3. ratio: interval with true absolute 0—0 means lack of attribute

4. interval: rating data of equal distances, 0 means something

Are IV and DV continuous or categorical variables?

• IV: categorical

• DV: continuous (usually normally distributed)

Some key points about ANOVA:

• Most used statistical method by experimental psychologists

• categorical IVs (2+ categories)

• can have more than 1 IV

• a single continuous DV

• usually assumed normally distributed

what are the measures of central tendency?

• mean: most common

  • vulnerable to extreme values(outliers)

• median: less vulnerable to extreme values (used when they are present)

• mode:

  • unaffected by extreme values

  • used for numerical or categorical data (but mainly categorical/ nominal data)

  • may be none

  • may be several

what are the measures of variation?

• range: simplest measure of dispersion

• variance: SS/n-1 (n-1: less bias and closer to pop)

• standard deviation: most common for descriptive stats

  • shows variation about the mean

  • same units as original data

  • square root of variance

  • the larger the standard deviation, the more variability

what are some ways to interpret variance and standard deviation better?

• SD converted to z-score= better

• if distribution is normal, SD can tell us about how many scores are above/ below a particular score

what is kurtosis?

the peak of the distribution

skewness:

• left (negative) skew: mean < median

• symmetric: mean=median

• right (positive) skew: median > mean

what are the characteristics of a normal distribution?

• DV expected to be continuous and normally distributed

• mean=median=mode

• population mean and standard deviation are sufficient to describe a normal distribution

  • contain about 68% of the values in pop

• mean ± 2 SD = 95% of values in population or sample

• mean ± 3 SD= 99.7% of values in population or sample

Hypotheses are assumptions about ___

population parameters NOT sample statistics (ex. studying 10 hours leads to better results than studying only 5 hours)

what are the steps for hypothesis testing (NHST)?

1. set up a hypothesis (H0 and H1)

2. choose alpha level, aka the cutoff value/critical value (0.05 unless told otherwise)

3. examine data and decide which statistical test to use (z,t,f)→ know when these are used

4. make a decision whether to reject or not reject the null hypothesis (i.e. whether the result is significant enough)

what does the p-value represent?

the probability of finding an effect in the observed statistic under the assumption that the null hypothesis is true

• if p is less than or equal to a, reject the null

a statistically significant effect does not mean that…

1. we have a precise estimate of the effect (errors, effect of pop maybe smaller or larger than estimate, standard error of estimate)

2. the effect is important or meaningful (depending on the scale—the cutoff, it helps us determine whether there is a significant effect given our dataset)

what does a confidence interval give us information about?

the precision of our estimates

• a CI should contain the population parameter but this doesn’t mean it always will

How does sample size influence the precisions of our estimates?

• larger sample= more precise

• smaller sample= less precise

• CI beocmes more narrow =more precise

What happens to the CI as alpha decreases?

the CI becomes larger or wider (less precise)

What are some commonly used effect size measure in NHST?

• pearson’s correlation r or r squared

• cohen’s d

• omega or omega squared

• eta squared

what are considered small, medium and large effect sizes for pearson and cohen's d?

small: r=0.10, r² = 0.01, d= 0.2

medium: r=0.30, r²=0.09, d= 0.5

large: r= 0.50, r²= 0.25, d= 0.8

what are the two types of errors in hypothesis testing and define them

• type 1 error (a): rejecting null hyp when it is true (false positive)

• type 2 error (beta): retaining null hyp when it is false (false negative)

what is power (1-beta)

The probability of correctly rejecting a false H0

what is the relationship between alpha and beta?

• Higher values of alpha (type 1 error) means lower values of beta (type 2 error), which means the power to reject the null is higher

• the two are inversely related

what information do you need to compute a z-test

• know population standard deviation or sample, sample size, and population mean or mean differences (x1/x2)

what is the purpose of a single mean z-test?

test whether the population mean is equal to some hypothesized value based on the sample mean that we have

what are the assumptions of a single mean z-test?

• normal distribution

• sample is simple and random (independence of observations)

• the population standard deviation must be known

z score for sample score formula

z=x-μ/σ

what is the formula for z-statistic for sample mean?

z= x̄- μ0/σx̄

• σx̄ : sample standard deviation (i.e. standard error)

• σx̄= σ/√N

• x̄: sample mean

• μ0= population mean

what are the limitations of the z-test?

• knowing the true population standard deviation is unrealistic (unless the entire population is known)

• this is why t-tests are alternatives to z-tests

what is the purpose of a t-test?

to test whether the population mean is equal to some hypothesized value based on the sample mean

** we have no information about the population standard deviation

what are the assumptions/ requirements of t-tests?

• the variable is normally distributed

• the sample is simple and random (independence of observations—in no way influenced by measurements of other subjects)

what is the formula t-test statistic?

t= x̄- μ0/sx̄

• μ0: hypothesized population mean (i.e. 0)

• x̄: sample mean

• sx̄= s/sqrt N (standard error)

who was the t distribution discovered by?

William S. Gosset in 1908 (a.k.a “student’s t distribution)

How does the t distribution vary in shape?

• the shape varies according to the degrees of freedom (N-1)

  • + sample size= +df

• distributions are quite close to the normal distribution for df>30

• as N +, t approaches z

Purpose of t-test with 2 means

To test whether 2 unknown population means (µ₁ and µ₂) are different from each other

• H0: µ₁ = µ₂

• H1: µ₁ ≠ µ₂

• the two samples may be independent or correlated

What are independent samples/ between subject designs?

Each participant goes through 1 of the conditions in the experiment

What is a correlated sample/ dependent/within-subjects design/ repeated measures?

Each participant in the sample goes through all of the conditions in the experiment (not the same as independent samples)

what are the requirements for independent samples t-tests?

• normally distributed in both populations

• the standard deviations of the populations are the same (i.e. homogeneity of variance)

• each subject is independent (simple, random sample

formula for independent samples t-test

t=x̄1-x̄2/ Sx̄1-x̄2=Dbar/SDbar

• in other words, you calculate the mean difference of both group and subtract it by 0(null), then you divide it by the standard error of the mean difference

• SDbar= pooled variance

• Dbar= x̄1-x̄2

How do you calculate the SDbar or Sx̄1-x̄2 in the independent samples t-test

1. calculate the SS for each group

2. use formula: S²P=SS1+SS2/df1 + df2

3. find S.E: sqrt (S²P/n1 + S²P/n2)

4. calculate t-statistic

5. find the effect size using cohen’s d

**Df=N1-1 + N2-1

what is the formula for cohen’s d?

d=x̄1-x̄2/s

the mean difference divided by the the pooled standard deviation

• s= pooled or estimated standard deviation (i.e. sqrt of s²)

**cohen’s d is most commonly used in t-tests

How do we calculate r using t-observed?

r=sqrt(t²/t²+df)

what is r²?

The proportion of variance in the Dependent variable that is explained by the independent variable