11: Null Hypothesis Significance Testing

frequentism

what would happen if we did this an infinite number of times?

• random phenomena have regular and predictable patterns in the long run

  • like random sampling error

probability

proportion of times outcome occurs in long run series of events (collective

• ranges from 0 (never occurs) to 1 (always occurs)

law of large numbers

more repetitions → mean proportion of observed outcomes will be closer to true probability

null hypothesis significance testing (NHST)

using probability to make inferences about population parameters using sample statistics

• use p-values to reject or accept the null hypothesis (Ho)

p-value

probability, assuming that Ho is true, that a sample statistic would be as extreme as more extreme than the obtained (your) sample statistic

• use cutoff value to control long-term Type I error rates

• cutoff value (a) has been arbitrarily set for us at .05

• if Ho is true, in the long-run, studies on this effect will incorrectly reject Ho 5% of the time

know that

• significance is NOT a property of populations

• decision rules are laid down before data are collected

• a more significant result does NOT mean a more important result or a larger effect size

• can encourage weak theorizing