Starnes, UPDATED The Practice of Statistics, 6e, Chapter 9

significance test

Formal procedure for using observed data to decide between two competing claims (the null hypothesis and the alternative hypothesis). The claims are usually statements about parameters.

null hypothesis H₀

Claim we weigh evidence against in a significance test. Often the null hypothesis is a statement of "no difference."

alternative hypothesis Hₐ

The claim that we are trying to find evidence for in a significance test.

one-sided alternative hypothesis

An alternative hypothesis is one-sided if it states that a parameter is greater than the null value or if it states that the parameter is less than the null value.

two-sided alternative hypothesis

The alternative hypothesis is two-sided if it states that the parameter is different from the null value (it could be either smaller or larger).

P-value

The probability of getting evidence for the alternative hypothesis Hₐ as strong as or stronger than the observed evidence when the null hypothesis H₀ is true. The smaller the P-value, the stronger the evidence against H₀ and in favor of Hₐ provided by the data.

significance level

Fixed value α that we use as a boundary for deciding whether an observed result is too unlikely to happen by chance alone when the null hypothesis is true. The significance level gives the probability of a Type I error.

Type I error

An error that occurs if we reject H₀ when H₀ is true. That is, the data give convincing evidence that Hₐ is true when it really isn't.

Type II error

An error that occurs if we fail to reject H₀ when Hₐ is true. That is, the data do not give convincing evidence that Hₐ is true when it really is.

standardized test statistic

Value that measures how far a sample statistic is from what we would expect if the null hypothesis H₀ were true, in standard deviation units.

power

The probability that a test will find convincing evidence for Hₐ when a specific alternative value of the parameter is true. The power of a test against any alternative is 1 minus the probability of a Type II error for that alternative; that is, power = 1 − P(Type II error).