Null Hypothesis Significance Testing

What is the purpose of NHST?

To determine whether it is likely or unlikely to obtain a sample statistic from a given population, assuming the null hypothesis is true.

What are the two components of inference in statistics?

1. Sampling distributions (tools)
2. NHST (rules for making decisions)

What is the "big idea" of NHST?

If a sample mean is improbable under the population distribution, it may fit better with an alternative hypothesis.

What are the simplified steps in a psychological experiment (NHST context)?

1. Sample individuals from a population

2. Administer a treatment/intervention

3. Measure outcomes

4. Compare to the known/assumed population

What is Step 1 of NHST?

State the population distribution:
e.g., 𝑋 ∼ 𝑁(7.20, 4.41)

What is Step 2 of NHST?

State null and alternative hypotheses (statistical form):

• 𝐻₀: μ = 7.20

• 𝐻₁: μ ≠ 7.20 (non-directional) or μ > 7.20 / μ < 7.20 (directional)

What does it mean to reject or fail to reject the null?

• Reject 𝐻₀: Sample mean is unlikely under null (suggests effect)

• Fail to reject 𝐻₀: Sample mean is not unlikely enough to suggest an effect

What is Step 3 of NHST?

State assumptions of the test, e.g., population is normally distributed.

What is Step 4 of NHST?

Identify the test statistic and deduce the null distribution
(e.g., use z = (𝑥̄ − μ) / (σ / √N) assuming 𝑁(0,1) under 𝐻₀)

What is a null distribution?

A sampling distribution of the test statistic assuming the null hypothesis is true.

What is Step 5 of NHST?

• Set alpha (𝛼)

• Determine critical value(s) (e.g., ±1.96 for α = .05 two-tailed)

• Establish decision rule (e.g., reject 𝐻₀ if z_observed > z_critical)

What is Step 6 of NHST?

Collect and analyze data, calculate the sample statistic, and check assumptions.

What is Step 7 of NHST?

Conduct the test

• Calculate z_obs

• Compare to z_crit or check p-value

• Make decision about 𝐻₀

What was the result of the Sleep 210 example?

z = 1.5 < z_crit = 1.65, so we fail to reject 𝐻₀. Sleep210 showed no statistically significant effect.

What does a p-value represent?

The probability of obtaining a test statistic as extreme or more extreme than the one observed, assuming 𝐻₀ is true.

What does alpha (𝛼) represent in NHST?

The threshold probability for rejecting the null hypothesis (commonly set at .05).

When do you use a one-tailed vs. two-tailed test?

• One-tailed: when expecting a specific direction (e.g., μ > 7.2)

• Two-tailed: when testing for any difference (μ ≠ 7.2)

What was the z-score in the salary example practice problem?

z = -2.24, which is statistically significant at p = .013. → Reject 𝐻₀