Inference for Proportions: Hypothesis Tests, Errors, and Two-Proportion Inference

Introduction to Significance Tests and Setting Up Hypotheses

When you do statistical inference, you’re using data from a sample to learn about a population. With proportions, the population parameter you care about is usually the true fraction of individuals in a population with some characteristic—called the population proportion.

A significance test (also called a hypothesis test) is a formal way to decide whether sample evidence is strong enough to support a claim about a population parameter. The core idea is simple:

• Start by assuming a “status quo” claim about the parameter.
• Ask: If that claim were true, how surprising is the sample result I got?
• If it would be very surprising, you have evidence against the status quo.

Parameters vs. statistics (what you know vs. what you estimate)

In inference, it’s crucial to separate:

• Parameter: a fixed (but usually unknown) value describing the population, like p.
• Statistic: a number computed from the sample, like the sample proportion \hat{p}.

If you observe x “successes” (people with the characteristic) in a sample of size n, then:

\hat{p} = \frac{x}{n}

The logic of hypothesis testing

A test is built around two competing statements:

• Null hypothesis: the default assumption; typically “no change” or “no difference.”
• Alternative hypothesis: what you’re looking for evidence to support.

You then compute a p-value, which is:

• p-value: the probability, assuming the null hypothesis is true, of getting a result at least as extreme as the one you observed.

Small p-values mean your observed result would be unlikely if the null were true, so the data provide evidence against the null.

Setting up hypotheses for a population proportion

Suppose a company claims that 40% of customers prefer Product A. Let p be the true proportion of all customers who prefer Product A.

A standard setup is:

• Null hypothesis: includes an equals sign and states a specific value.

H_0: p = 0.40

• Alternative hypothesis: reflects the research question and uses one of three forms.

Two-sided (detect any difference):

H_a: p \ne 0.40

Right-tailed (detect an increase):

H_a: p > 0.40

Left-tailed (detect a decrease):

H_a: p < 0.40

The alternative hypothesis should match the question before you look at the data. Choosing a one-sided alternative after seeing the sample result is a common (and serious) mistake.

Significance level and decisions

The significance level \alpha is the cutoff you choose for how strong the evidence must be to reject the null. Common choices are \alpha = 0.05 or \alpha = 0.01.

Decision rule:

• If p-value \le \alpha: **reject** H_0 (evidence supports H_a).
• If p-value > \alpha: **fail to reject** H_0 (not enough evidence for H_a).

“Fail to reject” is not the same as “accept.” You are not proving the null is true—you’re saying the sample didn’t provide strong evidence against it.

Notation snapshot (common in AP Statistics)

Example (hypotheses only)

A school believes more than 30% of students get at least 8 hours of sleep.

Let p = true proportion of all students at the school who get at least 8 hours of sleep.

H_0: p = 0.30

H_a: p > 0.30

That “greater than” comes directly from “more than 30%.”

Exam Focus

• Typical question patterns:
  • “A company claims that … Do the data provide convincing evidence that the true proportion is (greater/less/different)?”
  • “Write appropriate hypotheses for the parameter described.”
  • “Interpret the p-value in context.”
• Common mistakes:
  • Writing H_0: \hat{p} = 0.40 (null hypotheses are about p, not \hat{p}).
  • Using \ne when the question implies a one-sided alternative (or vice versa).
  • Saying “there is a 5% chance H_0 is true” (p-values do not give probabilities that hypotheses are true).

Carrying Out a Significance Test for a Population Proportion

For AP Statistics, the standard significance test for a single population proportion is the one-proportion z test. It uses the idea that when n is large enough, the sampling distribution of \hat{p} is approximately Normal.

When a Normal model is reasonable (conditions)

A hypothesis test is only as trustworthy as its conditions. For a one-proportion z test, you typically check:

1. Randomness: Data come from a random sample or a randomized experiment.
2. Independence (10% condition): If sampling without replacement, the sample size should be less than 10% of the population size.
3. Large counts (Normal approximation): Under the null hypothesis, both expected counts are at least 10:

np_0 \ge 10

n(1-p_0) \ge 10

Notice the test uses p_0 (the null value) in the condition, because the null hypothesis is what you assume when calculating how unusual the sample result is.

Test statistic (how far from the null, in standard errors)

If H_0: p = p_0 and your sample gives \hat{p} from sample size n, the test statistic is:

z = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}

Interpretation:

• The numerator \hat{p} - p_0 is the difference between what you saw and what the null claims.
• The denominator is the standard deviation of \hat{p} under the null, often called the standard error under the null.
• The result z tells you how many standard deviations your sample proportion is from the null value.

From z to a p-value

The p-value depends on the alternative hypothesis:

• If H_a: p > p_0 (right-tailed), p-value is the area to the right of your z.
• If H_a: p < p_0 (left-tailed), p-value is the area to the left.
• If H_a: p \ne p_0 (two-sided), p-value is **twice** the tail area beyond |z|.

You can find this using technology (calculator function, applet, or Normal CDF).

The AP Statistics “State, Plan, Do, Conclude” structure

Free-response questions often reward clear communication. A strong solution usually includes:

• State: Parameter, hypotheses, and significance level.
• Plan: Name the test and check conditions.
• Do: Compute \hat{p}, z, and the p-value.
• Conclude: Decision (reject/fail to reject) and a contextual conclusion about p.

Worked example: one-proportion z test

A city website claims that 60% of residents support a new public transit plan. A random sample of 200 residents finds 132 support it.

Let p = true proportion of all city residents who support the plan.

State (hypotheses)

H_0: p = 0.60

H_a: p \ne 0.60

(We’re checking whether the claim is wrong in either direction.)

Plan (conditions)

• Random: the problem states a random sample.
• 10% condition: 200 is presumably less than 10% of all residents (you’d state this assumption if population size isn’t given).
• Large counts using p_0 = 0.60:

np_0 = 200(0.60) = 120 \ge 10

n(1-p_0) = 200(0.40) = 80 \ge 10

So a one-proportion z test is appropriate.

Do (compute)

Sample proportion:

\hat{p} = \frac{132}{200} = 0.66

Test statistic:

z = \frac{0.66 - 0.60}{\sqrt{\frac{0.60(0.40)}{200}}}

Compute the standard error under the null:

\sqrt{\frac{0.60(0.40)}{200}} = \sqrt{\frac{0.24}{200}} = \sqrt{0.0012} \approx 0.0346

Then:

z \approx \frac{0.06}{0.0346} \approx 1.73

Because the alternative is two-sided, the p-value is:

\text{p-value} = 2P(Z \ge 1.73)

Using Normal probabilities, P(Z \ge 1.73) is about 0.0418, so p-value is about 0.0836.

Conclude

At \alpha = 0.05, p-value \approx 0.084 > 0.05, so we fail to reject H_0. The sample does not provide convincing evidence that the true proportion of residents who support the plan differs from 60%.

Notice the wording: you’re not saying “60% is true,” only that you don’t have strong evidence it’s different.

What can go wrong (common conceptual pitfalls)

• Using \hat{p} instead of p_0 in the test statistic’s standard error: In a test, you assume the null is true, so the standard error is based on p_0.
• Misinterpreting the p-value: A p-value is about the probability of the data (or more extreme) under H_0, not the probability H_0 is true.
• Ignoring direction: For a one-sided test, results in the “wrong” direction produce large p-values even if |z| looks big.

Exam Focus

• Typical question patterns:
  • “Do a significance test at \alpha = 0.05 and interpret the p-value.”
  • “Check conditions and identify the correct inference procedure.”
  • “Given a computer output with z and p-value, write the conclusion in context.”
• Common mistakes:
  • Checking large counts with \hat{p} instead of p_0 for a test.
  • Concluding “reject H_0, so p = p_0 is false” without stating what you have evidence for (the alternative, in context).
  • Treating “not significant” as “no effect” rather than “not enough evidence with this sample.”

Type I and Type II Errors and Power

Whenever you make a decision from sample data, you risk being wrong. Hypothesis testing organizes these risks into two types of errors.

The two error types (defined in plain language)

A hypothesis test ends in one of two decisions: reject H_0 or fail to reject H_0. Reality also has two possibilities: H_0 is true or H_0 is false. That creates four outcomes.

• Type I error: You reject H_0 even though H_0 is true.

  • In words: a “false alarm.”
  • Probability: approximately \alpha (the significance level), assuming conditions hold.

• Type II error: You fail to reject H_0 even though H_0 is false.

  • In words: you “miss” a real difference.
  • Probability: often called \beta (depends on the true value of p, the sample size, and \alpha).

• Power: the probability that you correctly reject H_0 when H_0 is false.

\text{Power} = 1 - \beta

Power matters because a test that almost never rejects H_0 isn’t very useful—even if it rarely makes Type I errors.

Why Type I and II errors depend on context

The labels “Type I” and “Type II” don’t automatically mean “worse” or “better.” The consequences depend on the setting.

Example context: testing whether a restaurant’s claim “60% of customers are satisfied” is accurate.

• Type I error (rejecting a true claim): you might publicly accuse the restaurant of misrepresenting satisfaction when it isn’t.
• Type II error (missing a false claim): you might fail to detect that satisfaction is actually lower, and customers keep getting misled.

In medical screening, the stakes can flip—false positives vs. false negatives have very different costs.

The tradeoff between \alpha and \beta

If you make it harder to reject H_0 (smaller \alpha), you reduce Type I error risk—but you often increase Type II errors (lower power), because you require stronger evidence to reject.

If you make it easier to reject H_0 (larger \alpha), you increase false alarms but reduce misses.

So you can’t usually minimize both error types at once without changing something else.

How to increase power (without “cheating”)

You increase power when it becomes easier for the test to detect real differences. Common ways:

1. Increase sample size n: this reduces the standard error, so real differences create larger |z| values.
2. Use a larger significance level \alpha: easier to reject H_0, but increases Type I error risk.
3. Have a true parameter farther from the null: if the true p is very different from p_0, the test will detect it more often.
4. Reduce variability: for proportions, variability is tied to p(1-p), though you typically don’t control this directly.

Example: describing Type I and Type II errors in context

A manufacturer tests:

H_0: p = 0.02

H_a: p > 0.02

where p is the proportion of products that are defective. They reject H_0 if the sample provides strong evidence the defect rate is higher than 2%.

• Type I error (in context): Conclude the defect rate is higher than 2% when in reality it is 2%.
• Type II error (in context): Fail to detect that the defect rate is higher than 2% when in reality it is higher than 2%.

This “put it in words” skill is heavily tested.

A simple power calculation idea (conceptual, with one numeric illustration)

Power calculations can be done using Normal approximations. The underlying idea is:

• Under H_0, \hat{p} is centered at p_0.
• Under a specific alternative value (say p = p_1), \hat{p} is centered at p_1.
• A rejection region (based on \alpha) cuts off extreme values.
• Power is the probability that \hat{p} falls in the rejection region when p = p_1.

Illustration (one-sided): Suppose you test

H_0: p = 0.50

H_a: p > 0.50

with n = 100 and \alpha = 0.05. For a right-tailed z test, the critical z value is about 1.645. That corresponds to rejecting when

\hat{p} > p_0 + 1.645\sqrt{\frac{p_0(1-p_0)}{n}}

Compute the cutoff:

\sqrt{\frac{0.50(0.50)}{100}} = \sqrt{0.0025} = 0.05

So reject when:

\hat{p} > 0.50 + 1.645(0.05) = 0.58225

If the true proportion were actually p = 0.62, then power is approximately:

P(\hat{p} > 0.58225 \mid p = 0.62)

Using a Normal model under the alternative:

\hat{p} \approx N\left(0.62, \sqrt{\frac{0.62(0.38)}{100}}\right)

The standard deviation under p = 0.62 is:

\sqrt{\frac{0.62(0.38)}{100}} = \sqrt{0.002356} \approx 0.0485

Compute the z-value for the cutoff under the alternative:

z = \frac{0.58225 - 0.62}{0.0485} \approx -0.78

So power is approximately P(Z > -0.78), which is about 0.78. Interpreting that: if the true proportion is 0.62, this test would reject H_0 about 78% of the time.

You’re not always asked to compute power numerically on AP, but you are expected to understand what affects it and how to describe errors.

Exam Focus

• Typical question patterns:
  • “Describe a Type I error and a Type II error in context for this test.”
  • “If the significance level is lowered, what happens to Type I error risk and power?”
  • “How could you increase the power of this test?”
• Common mistakes:
  • Describing errors without context (you must say what wrong conclusion you reached about p).
  • Thinking \alpha is the probability of a Type I error no matter what (it’s set as the long-run rate when H_0 is true).
  • Claiming that increasing n reduces both Type I and Type II errors automatically (Type I error is controlled by \alpha, but power typically increases with n).

Confidence Intervals and Tests for the Difference of Two Proportions

Inference for proportions often comes in two closely related forms:

• Confidence intervals estimate a parameter.
• Significance tests assess evidence for a claim.

They are connected: a two-sided test at significance level \alpha lines up with a confidence interval at confidence level 1-\alpha.

How confidence intervals connect to significance tests (one proportion)

A confidence interval gives a range of plausible values for p. For a one-proportion z interval, the standard form is:

\hat{p} \pm z^*\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}

Key point: the interval uses \hat{p} in the standard error, not p_0. That’s because intervals estimate based on what the sample suggests.

Connection to a two-sided test:

• If a 100(1-\alpha)\% confidence interval for p **does not include** p_0, then a two-sided test at level \alpha would reject H_0: p = p_0.
• If it does include p_0, you would fail to reject at that \alpha.

This connection is extremely useful for interpretation, but be careful: it aligns most cleanly for two-sided tests.

Moving to two proportions: what changes?

Often you want to compare two groups: new vs. old method, treatment vs. control, Group A vs. Group B.

Define parameters:

• p_1 = true proportion of successes in population 1
• p_2 = true proportion of successes in population 2

The parameter of interest is usually:

p_1 - p_2

From two independent samples:

• Sample 1: x_1 successes out of n_1, so \hat{p}_1 = \frac{x_1}{n_1}
• Sample 2: x_2 successes out of n_2, so \hat{p}_2 = \frac{x_2}{n_2}

Conditions for two-proportion inference

You typically check:

1. Random: two random samples, or a randomized experiment with two groups.
2. Independent groups: the two samples/groups are independent (no matching pairs here).
3. 10% condition: each sample is less than 10% of its population if sampling without replacement.
4. Large counts: expected success/failure counts are at least 10 in each group.

For a confidence interval, you check counts using \hat{p}_1 and \hat{p}_2:

n_1\hat{p}_1 \ge 10

n_1(1-\hat{p}_1) \ge 10

n_2\hat{p}_2 \ge 10

n_2(1-\hat{p}_2) \ge 10

For a significance test of H_0: p_1 = p_2, you often check large counts using a pooled estimate (explained next).

Confidence interval for p_1 - p_2 (two-proportion z interval)

The standard error for an interval uses separate sample proportions:

SE = \sqrt{\frac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \frac{\hat{p}_2(1-\hat{p}_2)}{n_2}}

Then the interval is:

(\hat{p}_1 - \hat{p}_2) \pm z^* SE

Interpretation tip: A confidence interval for p_1 - p_2 that is entirely positive suggests p_1 > p_2; entirely negative suggests p_1 < p_2.

Significance test for p_1 - p_2 (two-proportion z test)

A common null hypothesis is:

H_0: p_1 - p_2 = 0

which is equivalent to H_0: p_1 = p_2.

Under the null, it makes sense to assume both groups share a common true proportion, so you estimate that common value using the pooled proportion:

\hat{p} = \frac{x_1 + x_2}{n_1 + n_2}

Then the standard error under H_0 is:

SE_0 = \sqrt{\hat{p}(1-\hat{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}

And the test statistic is:

z = \frac{(\hat{p}_1 - \hat{p}_2) - 0}{SE_0}

This “pooled vs. unpooled” distinction is one of the most tested technical details:

• Intervals: unpooled standard error (uses \hat{p}_1 and \hat{p}_2).
• Tests with H_0: p_1 = p_2: pooled standard error (uses \hat{p}).

Worked example: two-proportion z interval and test

A school tries a new email reminder system to reduce late homework.

• Group 1 (new system): 40 of 200 students turned in late homework at least once.
• Group 2 (old system): 65 of 210 students turned in late homework at least once.

Let p_1 be the true proportion of students who would be late with the new system, and p_2 with the old system.

(A) Confidence interval for p_1 - p_2

Compute sample proportions:

\hat{p}_1 = \frac{40}{200} = 0.20

\hat{p}_2 = \frac{65}{210} \approx 0.3095

Difference:

\hat{p}_1 - \hat{p}_2 \approx 0.20 - 0.3095 = -0.1095

Standard error (unpooled):

SE = \sqrt{\frac{0.20(0.80)}{200} + \frac{0.3095(0.6905)}{210}}

Compute pieces:

\frac{0.20(0.80)}{200} = \frac{0.16}{200} = 0.0008

\frac{0.3095(0.6905)}{210} \approx \frac{0.2137}{210} \approx 0.001018

So:

SE \approx \sqrt{0.0008 + 0.001018} = \sqrt{0.001818} \approx 0.0426

For a 95% confidence interval, use z^* \approx 1.96:

(\hat{p}_1 - \hat{p}_2) \pm 1.96(0.0426)

Margin of error:

1.96(0.0426) \approx 0.0835

Interval:

-0.1095 \pm 0.0835

Approximately:

( -0.193, -0.026 )

Interpretation: You are 95% confident the true proportion late under the new system is between 2.6 and 19.3 percentage points lower than under the old system.

(B) Significance test for p_1 - p_2

Suppose you test whether the new system reduces late homework:

H_0: p_1 = p_2

H_a: p_1 < p_2

Pooled proportion:

\hat{p} = \frac{40 + 65}{200 + 210} = \frac{105}{410} \approx 0.2561

Standard error under null:

SE_0 = \sqrt{0.2561(0.7439)\left(\frac{1}{200} + \frac{1}{210}\right)}

Compute:

0.2561(0.7439) \approx 0.1905

\frac{1}{200} + \frac{1}{210} \approx 0.005000 + 0.004762 = 0.009762

So:

SE_0 \approx \sqrt{0.1905(0.009762)} = \sqrt{0.001859} \approx 0.0431

Test statistic:

z = \frac{-0.1095}{0.0431} \approx -2.54

Left-tailed p-value is P(Z \le -2.54), which is about 0.0055.

Conclusion at \alpha = 0.05: p-value is much smaller than 0.05, so reject H_0. There is convincing evidence that the new email reminder system reduces the proportion of students who turn in late homework.

Notice how the confidence interval and test agree: the interval for p_1 - p_2 was entirely negative, consistent with rejecting H_0: p_1 - p_2 = 0.

What can go wrong in two-proportion problems

• Mixing up pooled and unpooled standard errors: pooled for tests (when null says equal), unpooled for intervals.
• Confusing independence: two-proportion z methods assume independent groups. If the same individuals are measured twice or matched, that’s a different procedure (matched pairs).
• Interpreting a CI backwards: a 95% CI does not mean “95% of individuals are in this range.” It’s about plausible values of the parameter.

Exam Focus

• Typical question patterns:
  • “Construct and interpret a confidence interval for p_1 - p_2.”
  • “Test whether the proportions differ (or whether one is larger) using a two-proportion z test.”
  • “Use the confidence interval to assess a claim about a difference in proportions.”
• Common mistakes:
  • Using the pooled proportion in a confidence interval standard error.
  • Forgetting to define p_1 and p_2 in context (AP scoring often requires parameter definition).
  • Concluding causation from two samples when the design is observational (causal language is best reserved for randomized experiments).