Chapter 9: Testing a Claim - Significance Tests

Chapter 9: Testing a Claim

Section 9.1: Significance Tests: The Basics

Learning Targets

  • STATE appropriate hypotheses for a significance test about a population parameter.

  • INTERPRET a P-value in context.

  • MAKE an appropriate conclusion for a significance test.

  • INTERPRET a Type I error and a Type II error in context, providing a consequence of each error in a given setting.

Stating Hypotheses

  • Confidence intervals and significance tests are the two most common methods of statistical inference.

    • Confidence Intervals: Provide a range of values for the population parameter based on sample data.

    • Significance Tests: Weigh evidence for or against a particular claim using observed data.

The Process of a Significance Test

  • A significance test (or hypothesis test) is a formal procedure used to decide between two competing claims, known as hypotheses.

    • null hypothesis (H0): The claim that we weigh evidence against (e.g., no effect or no difference).

    • alternative hypothesis (Ha): The claim we are trying to find evidence for (indicating some effect or difference).

    • Example: A free-throw shooter claims that his proportion of made free throws is $p = 0.8$ ; we suspect exaggeration, setting:

    • Null hypothesis: $H0: p = 0.80$

    • Alternative hypothesis: $Ha: p < 0.80$

Types of Alternative Hypotheses

  • The alternative hypothesis can be:

    • One-Sided: Indicates that a parameter is either greater than or less than the null value.

    • Examples:

      • $H0: p = 0.80$, $Ha: p < 0.80$ (less than)

      • $H0: p = 0.80$, $Ha: p > 0.80$ (greater than)

    • Two-Sided: States that a parameter is different from the null value (could be either direction).

    • Example: $H0: p = 0.80$, $Ha: p
      eq 0.80$

Defining Hypotheses in Context

  • Practical application: For the Hawaii Pineapple Company, where the mean weight of pineapples last year was 31 ounces, with a new irrigation system installed:

    • Appropriate hypotheses:

    • $H0: µ = 31$ (null hypothesis: mean weight remains the same)

    • $Ha: µ
      eq 31$ (alternative hypothesis: mean weight has changed)

    • where $µ$ = the true mean weight of all pineapples grown in the field this year.

Importance of Hypotheses

  • Hypotheses should express beliefs or suspicions prior to analyzing data.

  • Always refer to population parameters, not sample statistics.

    • Incorrect: $H0: ar{p}= 0.80$ or $Ha: ar{x}= 31$

Interpreting P-values

  • A player claiming to make $80\%$ of his free throws makes only $ar{p} = 0.64$ in a random sample of 50 trials.

    • This serves as evidence against $H0: p = 0.80$ and supports $Ha: p < 0.80$.

  • P-value Definition: Probability of observing evidence as strong or stronger than the current evidence if $H0$ is true.

  • Example numerical calculation:

    • $P-value ext{ approx } rac{3}{400} = 0.0075$

    • Indicates how likely it is for an $80\%$ shooter to achieve $64\%$ or lower simply by chance.

Hypothetical Research Problem: Calcium Intake

  • The NIH recommends $1300\text{ mg}$ daily for teenagers.

  • A test is constructed with:

    • Null hypothesis: $H0: µ = 1300$

    • Alternative hypothesis: $Ha: µ < 1300$

    • Sample: 20 teens report average intake $ar{x} = 1198$ mg, with standard deviation $s_x = 411$ mg.

  • Resulting P-value obtained: $0.1404$.

    • Interpretation:

    • (a) If $H0$ is true, average intake is $1300$ mg.

    • (b) If average intake is truly $1300$ mg, there's a $0.1404$ probability of getting a sample average of $1198$ mg or less by random chance.

Making Conclusions from P-values

  • Decisions in significance tests based on P-value strength:

    • Reject $H0$ if the result is unlikely due to chance alone (small P-value).

    • Fail to reject $H0$ if the result is not unlikely (large P-value).

Guidelines for Drawing Conclusions

  • If $P-value < 0.05$, reject $H0$ and conclude evidence for $Ha$.

  • If $P-value \, ext{ is not small}$, fail to reject $H0$ (not convincing evidence for $Ha$).

Significance Level (α)

  • Significance level $ ext{α}$ defines the boundary for determining statistical significance.

    • Common levels are $ ext{α = 0.05}$, $ ext{α = 0.01}$, and $ ext{α = 0.10}$.

  • Preference varies depending on whether Type I or II errors are more serious:

    • Type I Error: Rejecting $H0$ when it's true.

    • Probability is equal to $ ext{α}$.

    • Type II Error: Failing to reject $H0$ when $Ha$ is actually true.

Type I and II Errors in Context

  • Example: Potato chip production scenario, testing for blemishes in potatoes.

    • Type I Error: Concludes >8% blemished when it’s really 8%.

    • Consequence: Rejects good shipment, wasting resources.

    • Type II Error: Fails to find evidence for >8% blemishes when they exist.

    • Consequence: Produces bad chips, harms reputation and reduces sales.

Summary

  • Hypotheses define the framework for significance tests.

  • P-values assist in interpreting evidence against the null hypothesis.

  • Conclusions guide decision-making based on statistical significance levels and the implications of Type I and Type II errors.