Statistical Significance & Effect Size

what occurs in null hypothesis statistical testing (NHST)?

binary decisions (reject or retain H₀) based on probabilistic information

how is statistical significance tested?

determine probability of obtaining an F ratio this extreme under the H₀

what are the two alternatives of statistical significance?

probability < .05, reject H₀ or probability > .05, retain H₀

what is the assumption of NHST?

there is no effect or no relationship between the variables being studied

how are there errors in hypothesis testing?

never know if H₀ is true or not, so deciding to reject or retain could be wrong

what are the two errors in hypothesis testing?

type I & type II errors

what is a type I error?

when H₀ is true but we reject it; failed to retain H₀; false positive, incorrect rejection

how frequently are type I errors likely to be committed?

depends on F_critical (α); if α = .05 then 5% of the time

what is a type II error?

when H₀ is not true but we retain it; failed to reject; false negative, miss

how frequently are type II errors likely to be committed?

harder to determine frequency due to sampling distribution of F under H₁; β (probability of making a type II error ~.20%)

what is power?

the probability of not making a type II error; rejecting H₀ when it is not true

what happens when α is decreased?

decreases F value, decreases type I error but increases type II error

what happens when α is increased?

increases F value, decreases type II error but increases type I error; cannot increase α since .05 is the rule

what does the effect size do?

estimates potency or magnitude of an effect

what does eta squared represent?

the proportion of total variance in the DV that is explained by the IV; called the correlation ratio (R²)

what are the values of effect size?

values range between 0 & 1; 0 = IV explains none of the variance & 1 = IV explains all of the variance

what are the effect size classifications?

~.01 small effect (difficult to detect); ~.06 medium effect (large enough to be visible); ~.14 large effect (grossly perceptible)