Comprehensive Guide to Hypothesis Testing: Errors, Critical Values, Proportions, and Means

Foundations of Hypothesis Testing: Type I and Type II Errors

  • Hypothesis testing evaluates competing statements regarding population parameters using sample data: the null hypothesis (H0H_0) and the alternative hypothesis (H1H_1 or HaH_a).

  • Definition of a Type I Error:

    • Occurs strictly when the conclusion of a statistical hypothesis test is to reject the null hypothesis (H0H_0), but that decision constitutes a mistake.

    • Conceptual shorthand: A "false reject."

    • Underlying reality: In the real population, the null hypothesis (H0H_0) is actually true, valid, and accurate. The researcher erroneously rejects a true hypothesis.

  • Definition of a Type II Error:

    • Occurs strictly when the conclusion of a statistical hypothesis test is to fail to reject the null hypothesis (H0H_0), but that decision constitutes a mistake.

    • Conceptual mechanism: The test retains or fails to reject H0H_0 when H0H_0 should have been rejected in favor of the alternative hypothesis (H1H_1).

    • Underlying reality: In the real population, the null hypothesis (H0H_0) is false, and the alternative hypothesis (H1H_1) is the accurate state of nature.

  • General Problem-Solving Strategy for Multiple-Choice Error Questions:

    • Step 1: Write down the null hypothesis (H0H_0) and alternative hypothesis (H1H_1) based on the claim.

    • Step 2: Determine the required test decision based on the error type requested:

    • For a Type I error, the test decision must be to reject H0H_0.

    • For a Type II error, the test decision must be to fail to reject H0H_0.

    • Step 3: Eliminate the two answer options that state the incorrect test decision.

    • Step 4: Evaluate the remaining two choices by assessing the stated reality:

    • For Type I, the reality must be that H0H_0 is actually true.

    • For Type II, the reality must be that H1H_1 is actually true (H0H_0 is false).

Step-by-Step Error Identification Procedures

  • Example: Type I Error Identification (Cereal Box Weights):

    • Scenario: A cereal company claims that the mean weight of cereal in its packets is 14 ounces14\,\text{ounces}.

    • Hypotheses Setup:

    • Null hypothesis (H0H_0): μ=14 ounces\mu = 14\,\text{ounces} (contains the claim).

    • Alternative hypothesis (H1H_1): μ≠14 ounces\mu \neq 14\,\text{ounces}.

    • Test Decision Requirement:

    • A Type I error requires the decision to reject the null hypothesis claim (μ=14 ounces\mu = 14\,\text{ounces}).

    • Option Elimination:

    • Eliminate choices stating "fail to reject the claim that the mean weight is 14 ounces14\,\text{ounces}".

    • Eliminate choices stating "reject the claim that the mean weight is different from 14 ounces14\,\text{ounces}" (rejection applies to the null claim, not the alternative).

    • Reality Verification:

    • A Type I error requires rejecting H0H_0 when H0H_0 is true.

    • Correct description: Rejecting the claim that the mean weight is 14 ounces14\,\text{ounces} when the mean weight is actually equal to 14 ounces14\,\text{ounces}.

    • Incorrect alternative: Rejecting the claim when the mean is actually greater than 14 ounces14\,\text{ounces} represents a correct statistical decision, not a Type I error.

  • Example: Type II Error Identification (Adult Extreme Shyness):

    • Scenario: A psychologist claims that more than 2.4%2.4\% of adults suffer from extreme shyness.

    • Parameter and Hypotheses Setup:

    • The parameter involves a percentage or proportion (pp).

    • Alternative hypothesis (H1H_1): p>2.4%p > 2.4\% (or p>0.024p > 0.024) (contains the claim).

    • Null hypothesis (H0H_0): p=2.4%p = 2.4\% (or p=0.024p = 0.024).

    • Test Decision Requirement:

    • A Type II error requires the decision to fail to reject the null hypothesis (p=2.4%p = 2.4\%).

    • Option Elimination:

    • Eliminate choices stating "reject the hypothesis that the proportion is equal to 2.4%2.4\%".

    • Reality Verification:

    • A Type II error occurs when H0H_0 is false and H1H_1 is true in reality.

    • Correct description: Failing to reject the hypothesis that the percentage of adults who suffer from extreme shyness is equal to 2.4%2.4\% when the percentage is actually greater than 2.4%2.4\%.

    • Incorrect alternative: Stating the percentage is actually less than 2.4%2.4\% does not align with the condition where the specific alternative hypothesis (p>2.4%p > 2.4\%) is the true state of nature.

The Critical Value Method vs. The P-Value Method

  • Default Testing Paradigms:

    • The p-valuep\text{-value} method is the standard default approach across hypothesis tests.

    • The critical value method is employed whenever explicitly requested (e.g., instructions stating "using the critical value method" and "state your critical values").

  • Examination Reference Materials:

    • The critical value z-scorez\text{-score} chart is provided at the top of the examination.

    • Mathematical formulas are generally executed directly within calculator functions rather than calculated by hand, with the exception of calculating counts from proportions.

  • Converting Sample Proportions to Whole-Number Counts:

    • Formula for calculating the number of successes (xx):     x=p^×nx = \hat{p} \times n

    • Calculated values must be rounded to the nearest integer because graphing calculators will return a domain or syntax error if a non-integer value is provided for xx.

  • Graphical Decision Protocol for the Critical Value Method:

    • Sketch the standard normal distribution curve (z-distributionz\text{-distribution}) centered at 00.

    • Mark the critical value threshold (zcritz_{\text{crit}}) derived from the reference chart and shade the critical region (rejection region) in the appropriate tail(s).

    • Plot the test statistic (ztestz_{\text{test}}) relative to the critical value.

    • Decision rule:

    • If the test statistic falls inside the shaded critical region, reject H0H_0.

    • If the test statistic falls outside the shaded critical region (toward the center), fail to reject H0H_0.

    • Notation requirement: Always write the identifying letter (zz) alongside the numerical value for both the test statistic and critical value (e.g., z=2.35z = 2.35, z=2.326z = 2.326).

Single-Proportion Hypothesis Testing using the Critical Value Method

  • Case Study 1: Production Defect Rate (Right-Tailed Test):

    • Scenario: A manufacturer considers a production process "out of control" when defects exceed 3%3\%. In a random sample of n=185n = 185 items, the defect rate is p^=5.9%\hat{p} = 5.9\%. The manager claims this elevated defect rate is merely a sample fluctuation and production is not out of control.

    • Significance level: α=0.01\alpha = 0.01.

    • Operational definitions and hypotheses:

    • "Out of control": Defect rate exceeds 3%3\% (p>0.03p > 0.03).

    • Manager's claim: Process is not out of control (p=0.03p = 0.03).

    • Null hypothesis (H0H_0): p=0.03p = 0.03 (Claim).

    • Alternative hypothesis (H1H_1): p>0.03p > 0.03 (Right-tailed test).

    • Parameter and Distribution:

    • Parameter: Single population proportion (pp).

    • Distribution: Normal distribution.

    • Calculating Sample Count (xx):     x=p^×n=0.059×185=10.915x = \hat{p} \times n = 0.059 \times 185 = 10.915

    • Rounded to the nearest whole integer: x=11x = 11.

    • Calculator Function and Inputs:

    • Function: 1-PropZTest

    • Inputs: p0=0.03p_0 = 0.03, x=11x = 11, n=185n = 185, alternative condition: >p0> p_0.

    • Output and Critical Value Determination:

    • Test statistic: z=2.35z = 2.35.

    • Significance level: α=0.01\alpha = 0.01 for a right-tailed test.

    • Critical value from chart: z=2.326z = 2.326.

    • Visual Sketch and Decision:

    • Normal curve plotted with a shaded right-tail cutoff at critical value z=2.326z = 2.326.

    • Test statistic z=2.35z = 2.35 is plotted to the right of 2.3262.326, falling inside the shaded critical region.

    • Decision: Reject the null hypothesis (H0H_0).

    • Conclusion:

    • There is sufficient evidence to warrant rejection of the claim that the defect rate is due to sample fluctuation and production is not really out of control.

  • Case Study 2: Democratic Candidate Voter Preference (Left-Tailed Test):

    • Scenario: A poll of n=1068n = 1068 polled Americans revealed that 48%48\% voted for the Democratic candidate. Test the claim that fewer than half of all voters prefer the Democrat at the α=0.05\alpha = 0.05 significance level.

    • Numerical translation: "Half" translates directly to 0.50.5 (or 50%50\%).

    • Hypotheses Setup:

    • Null hypothesis (H0H_0): p=0.5p = 0.5.

    • Alternative hypothesis (H1H_1): p<0.5p < 0.5 (Claim, Left-tailed test).

    • Calculating Sample Count (xx):     x=0.48×1068=512.64x = 0.48 \times 1068 = 512.64

    • Rounded to the nearest whole integer: x=513x = 513.

    • Calculator Function and Inputs:

    • Function: 1-PropZTest

    • Inputs: p0=0.5p_0 = 0.5, x=513x = 513, n=1068n = 1068, alternative condition: <p0< p_0.

    • Output and Critical Value Determination:

    • Test statistic: z=−1.29z = -1.29.

    • Significance level: α=0.05\alpha = 0.05 for a left-tailed test.

    • Critical value from chart: z=−1.645z = -1.645.

    • Visual Sketch and Decision:

    • Normal curve plotted with a shaded left-tail cutoff at critical value z=−1.645z = -1.645.

    • The test statistic z=−1.29z = -1.29 lies between 00 and −1.645-1.645, falling outside the shaded critical rejection region.

    • Decision: Fail to reject the null hypothesis (H0H_0).

Two-Sample Proportion Tests and Confidence Intervals

  • Sample Independence Principle:

    • Two-proportion problems are invariably based on independent samples; proportions cannot be structured as dependent matched pairs.

  • Two-Sample Proportion Hypothesis Testing (Vaccine Effectiveness):

    • Scenario: 77 out of 85008500 people vaccinated against a disease later developed the disease; 1818 out of 1000010000 people vaccinated with a placebo later developed the disease. Test the claim that the vaccine is effective in lowering the incidence of disease at α=0.05\alpha = 0.05.

    • Sample Setup:

    • Vaccinated group: x1=7x_1 = 7, n1=8500n_1 = 8500.

    • Placebo group: x2=18x_2 = 18, n2=10000n_2 = 10000.

    • Hypotheses Setup:

    • Null hypothesis (H0H_0): p1=p2p_1 = p_2 (No difference between vaccine and placebo disease rates).

    • Alternative hypothesis (H1H_1): p1<p2p_1 < p_2 (Claim: Vaccine group rate is lower than placebo group rate; left-tailed test).

    • Calculator Function and Inputs:

    • Function: 2-PropZTest (Option 6 on the STAT TESTS menu).

    • Inputs: x1=7x_1 = 7, n1=8500n_1 = 8500, x2=18x_2 = 18, n2=10000n_2 = 10000, alternative condition: <p2< p_2.

    • Test Outputs:

    • Test statistic: z=−1.80z = -1.80.

    • p-valuep\text{-value}: 0.03580.0358 (rounded to four decimal places) or 0.0360.036 (rounded to three decimal places).

    • Decision and Conclusion Protocol:

    • Compare p-valuep\text{-value} to significance level: 0.0358<0.050.0358 < 0.05.

    • Decision rule: "If the pp is low, the null must go" - Reject H0H_0.

    • Conclusion: There is sufficient evidence to support the claim that the vaccine is effective in lowering the incidence of disease.

  • Two-Sample Proportion Confidence Intervals (Product Recognition):

    • Scenario: Of 558558 New Yorkers, 193193 recognized a product (x1=193x_1 = 193, n1=558n_1 = 558). In a second sample, 196196 out of 614614 recognized the product (x2=196x_2 = 196, n2=614n_2 = 614). Construct a 95%95\% confidence interval for the difference between population proportions (p1−p2p_1 - p_2).

    • Calculator Function Selection:

    • Proportion procedures always include Prop in the calculator title.

    • Function: 2-PropZInt (Do not use 2-SampTInt or ZInterval, which apply to means).

    • Input Order Rules:

    • Ensure the count of successes (xx) is entered first and the total sample size (nn) is entered second (x≤nx \le n).

    • Swapping xx and nn causes a calculator execution error.

    • Resulting Confidence Interval:

    • Structure: Format the parameter difference first as p1−p2p_1 - p_2.

    • Confidence interval bounds: Approximately −0.03<p1−p2<0.08-0.03 < p_1 - p_2 < 0.08 (or (−0.03,0.08)(-0.03, 0.08)).

    • Interpretation of Zero in Two-Sample Confidence Intervals:

    • If zero (00) is contained inside the confidence interval: There is no statistically significant difference between the two population parameters (p1p_1 and p2p_2).

    • If zero (00) is not contained inside the interval: A statistically significant difference exists between the two parameters.

    • Decision for this case: Because 00 falls inside the interval (−0.03,0.08)(-0.03, 0.08), there is no significant difference in product recognition between the two groups.

Two-Sample Hypothesis Testing for Means: Independent vs. Dependent Samples

  • Distinguishing Independent from Dependent (Paired) Samples:

    • Independent Samples: Sample selections are unrelated. Samples can have unequal sample sizes (n1≠n2n_1 \neq n_2). Even if sample sizes are equal, samples remain independent unless specific pairing criteria exist (e.g., identical twins, married couples, pre/post measurements on identical subjects).

    • Dependent Samples: Data points are inherently paired. Both lists must have equal lengths. Common indicator: "Before-and-after" designs measuring the same individuals across two time periods.

  • Independent Two-Sample Mean Testing (Employee Salaries):

    • Scenario: Weekly salaries are compared between n1=8n_1 = 8 female employees and n2=15n_2 = 15 male employees at α=0.05\alpha = 0.05. Test the claim that the mean salary of female employees is less than the mean salary of male employees.

    • Sample Independence: The sample sizes differ (8≠158 \neq 15), making pairing impossible; samples are independent.

    • Hypotheses Setup:

    • Null hypothesis (H0H_0): μ1=μ2\mu_1 = \mu_2.

    • Alternative hypothesis (H1H_1): μ1<μ2\mu_1 < \mu_2 (Claim).

    • Calculator Function: 2-SampTTest using the Data input mode with raw data in list L1L_1 and list L2L_2.

    • Quality-Control Verification Against Typos:

    • Entering raw lists carries a risk of data entry typographical errors.

    • When 2-SampTTest runs, the output displays the sample means (xˉ1\bar{x}_1 and xˉ2\bar{x}_2) calculated from the entered lists.

    • Benchmark checking: Verify that the calculated output means match the printed sample means provided on the test sheet (e.g., xˉ1=705.375\bar{x}_1 = 705.375 and xˉ2=817.07\bar{x}_2 = 817.07). Mismatched means indicate a list entry typo that must be corrected before submitting results.

  • Dependent Matched-Pairs Mean Testing (Diet Weight Reduction):

    • Scenario: Weights of 77 subjects measured before and after following a diet for two months at α=0.05\alpha = 0.05. Test the claim that the diet is effective in reducing weight.

    • Identifying Dependent Structure: The same 77 individuals are measured twice (before and after), establishing a matched-pair design.

    • Definition of Differences:

    • Let L1L_1 represent "Before" weights and L2L_2 represent "After" weights.

    • Define differences in list L3L_3 as:       L3=L1−L2L_3 = L_1 - L_2

    • Directional orientation: If the diet successfully reduces weight, the "Before" weight is larger than the "After" weight (L1>L2L_1 > L_2), resulting in a positive weight loss difference (L1−L2>0L_1 - L_2 > 0).

    • Hypotheses Setup (using mean difference notation μd\mu_d):

    • Null hypothesis (H0H_0): μd=0\mu_d = 0.

    • Alternative hypothesis (H1H_1): μd>0\mu_d > 0 (Claim: Effective weight reduction yields a positive difference when computed as Before minus After).

    • Calculator Function and Execution:

    • Populate L1L_1 with Before data and L2L_2 with After data.

    • Generate the difference list: L3=L1−L2L_3 = L_1 - L_2.

    • Run the single-sample T-Test directly on the difference list L3L_3.

    • Inputs: μ0=0\mu_0 = 0, Data list = L3L_3, alternative condition: >0> 0.

Confidence Intervals for Two Independent Means

  • Function Selection Protocol:

    • When estimating the difference between two independent population means (μ1−μ2\mu_1 - \mu_2), the appropriate calculator tool is 2-SampTInt.

    • Distinguishing Features:

    • 2-Samp specifies two distinct groups.

    • T denotes inference on means when population standard deviations are unknown.

    • Int designates a confidence interval calculation rather than a hypothesis test.

  • Execution Using Summary Statistics (Example 25):

    • Data inputs: Select Stats mode.

    • Enter sample statistics for both groups: sample means (xˉ1=365\bar{x}_1 = 365, xˉ2\bar{x}_2), sample standard deviations (s1=52s_1 = 52, s2s_2), and sample sizes (n1=185n_1 = 185, n2n_2).

    • Set the confidence level and calculate the interval.

  • Core Decision Rule for Two-Sample Confidence Intervals:

    • If zero (00) is included within the confidence interval: There is no statistically significant difference between the two population means (μ1\mu_1 and μ2\mu_2).

    • If zero (00) is not included within the confidence interval: There is a statistically significant difference between the two population means.

Student Questions and In-Class Clarifications

  • Clarification on Exam Formula Sheets and Reference Tables:

    • Question: Will a formula sheet be provided on the exam, or what aids will be available?

    • Clarification: The critical value z-scorez\text{-score} reference chart is the only printed reference table provided at the top of the examination. Formula sheets are not distributed because statistical calculations are executed via graphing calculator functions. The single manual calculation required is converting sample proportions to integer counts using x=p^×nx = \hat{p} \times n.

  • Clarification on Proportions and Sample Pairing:

    • Question: Can hypothesis tests for proportions ever involve dependent (paired) samples?

    • Clarification: No. Two-proportion procedures are exclusively conducted using independent samples. Dependent paired structures apply to mean comparisons (e.g., matched-pairs t-testst\text{-tests}).

  • Clarification on Calculator Input Errors:

    • Question: Why does the calculator display an error when running two-proportion tests or intervals?

    • Clarification: Calculator errors in two-proportion functions occur when the inputs for xx and nn are swapped. The value of xx represents the count of successes and must be less than or equal to the total sample size nn. Swapping the order results in a computational error.