11 inferential stats

null hypothesis significance testing (NHST)

1. formulate a set of hypotheses and assume the null hypothesis
2. collect data
3. calculate probability (*p-value*) and t-obt value
4. decide whether to reject or retain null hypothesis

null hypothesis (Ho)

baseline conservative assumption, IV doesn’t affect DV, no relationship between 2 variables

research hypothesis (H1 or Ha)

IV does affect DV, there IS a relationship between 2 variables, usually the hypothesis you hope will turn out to be true

mutually exclusive

cannot have both hypotheses be true at the same time

exhaustive

must account for all possible outcomes

alpha level

most common cut-off choice in psychology: 5%/0.05, if p < alpha then reject null hypothesis, vice versa

* big alpha = easier it is to find data consistent with research hypothesis, thus greater power

type i error

rejecting the null hypothesis when is it actually true (ie. false alarm, false positive)

publication bias

bias that emerges from the fact that statistically significant data is far more likely to be published than non-significant data

type ii error

retaining the null hypothesis when it is actually false (ie. miss, false negative)

power

probability of correctly rejecting the null hypothesis, ability to detect an effect if one truly exists

sample size

the greater the sample size the greater the power, less error in data to detect effect

magnitude of effect (effect size)

the larger the difference is in population, the easier it is to detect, therefore greater power → as effect size increase, type 2 error decreases, but type 1 error has NO change

william gosset

inventor of t-test, aka Student’s t-test

obtained values (t-obt)

a statistic that captures the r effect observed in your study → calculate t-obt in a t-test

t-obt equation

difference between means/variability in data

examples of noise

poorly worded questions, effect of uncontrolled variables, small sample size

degree of freedom

total number of participants minus number of groups → (n1 + n2 - 2)

* if mean is known then n-1 is the degree of freedom as the last value has to be fixed to match the mean

relationship between p-values & t-obt

p-value can be visualized as an amount of area in the t-distribution that corresponds to a particular value to t-obt, __the higher the t-obt, the lower the p-value = good!__

relationship between alpha level & critical value

alpha levels correspond to critical values (t-crit) the same way that p-values correspond to t-obt

if t-obt < t-crit

retain null hypothesis, p-value > alpha

if t-obt > t-crit

reject null hypothesis, p-value < alpha

p-hacking

set of thically questionable practices researcher use to get significant results, ie. keep adding participants to run study until p-value becomes statistically significant

\+ ways to p-hack

* toss out participants that disagree with your hypothesis
* use many measures of same construct, only report results that show significance
* find/report random interactions as though you predicted them, even if you didn’t

F-test (ANOVA)

extension of the t-test that can be used in many more research designs, usually used when there are more than 2 levels of an IV or when a factorial design with 2 or more IVs has been used

systematic variance

aka between-group variance, numerator in equation, deviation of the group means from the grand mean (mean score of all participants across all conditions)

error variance

aka within-group variance, denominator in equation, captures how much individual scores in each group deviate from respective group means