● Alpha level - The threshold P-value that determines when we reject a null hypothesis. If we observe a statistic whose P-value based on the null hypothesis is less than , we reject that null hypothesis. ● Statistically significant - When the P-value falls below the alpha level, we say that the test is “statistically significant” at that alpha level. ● Significance level - The alpha level is also called the significance level, most often in a phrase such as a conclusion that a particular test is “significant at the 5% significance level.”

● Type I error - The error of rejecting a null hypothesis when in fact it is true (also called a “false positive”). The probability of a Type I error is . ● Type II error - The error of failing to reject a null hypothesis when in fact it is false (also called a “false negative”). The probability of a Type II error is commonly denoted and depends on the effect size. ● Power - The probability that a hypothesis test will correctly reject a false null hypothesis is the power of the test. To find power, we must specify a particular alternative parameter value as the “true” value. ● Effect size - The difference between the null hypothesis value and true value of a model parameter is called the effect size.