Module 9: Uses and Abuses of Tests

What is statistical significance?

If the p-value is as small or smaller than α, then we say that the data are statistically significant at level α.

What is practical significance?

Refers to the magnitude of the effect size between the sample statistic and the hypothesized value of the parameter under Ho is large enough to matter

How do you calculate effect size for categorical data?

|𝑝̂ - 𝑝0|

How do you calculate effect size for quantitative data?

|𝑥̅ − 𝜇0|

What is Type I error?

Rejecting the null hypothesis when it is true.

𝛼 = 𝑃(𝑇𝑦𝑝𝑒 𝐼 𝐸𝑟𝑟𝑜𝑟)

What is Type II error?

Failing to reject the null hypothesis when it is false.

𝛽 = 𝑃(𝑇𝑦𝑝𝑒 𝐼𝐼 𝐸𝑟𝑟𝑜𝑟)

What does power mean?

Probability that the test will reject the null hypothesis at a chosen significance level 𝛼 when the specified alternative value of the parameter is true.

How do you calculate power?

Power = 1 - 𝛽

How do you increase the power of a test?

Increase significance level, sample size, and effect size. Decrease probability of Type II error and standard error.

What factors are required to interpret power?

Power given or stated as a probability, null is false, and reject the null.