Week 8: Hypothesis Testing and Comparing Populations

Comparing Populations: Foundational Concepts and Motivations

• Rationale for Comparing Populations:

  • Comparing populations allows for the understanding of differences and similarities in key characteristics beyond single-population parameters.

  • Central location measures, specifically means and proportions, are the primary metrics used to compare population characteristics.

  • Statistical inference relies on the fact that sample means and sample proportions are unbiased estimators of their respective population parameters.

• Practical Examples of Population Comparison:

  • Education: Comparing if mid-semester tasks result in higher marks than the final exam.

  • Product Development: Evaluating if a newly introduced product performs better than an existing one.

  • Economics: Investigating if individuals in metropolitan areas (cities) earn higher wages than those in regional areas.

  • Healthcare: Determining if a medical intervention leads to improved patient outcomes.

  • Public Opinion: Comparing if different opinion polls yield consistent results.

Experimental Designs: Independent vs. Matched Pairs Samples

• Independent Samples:

  • In this design, the two samples are collected completely independently of one another.

  • Example 1 (Wages): Comparing average wages between people in Sydney and Melbourne involves taking a random sample from Sydney and a separate random sample from Melbourne.

  • Example 2 (Education): Testing if tutor feedback improves reports by taking one random sample of students for the draft report and a completely different random sample for the final report. This may not be the most efficient method due to high variability between different students.

• Matched Pairs (Paired Samples):

  • This design involves collecting one random sample and observing every individual under two different conditions or time points.

  • Example 1 (Education): Comparing the draft and final report marks of the exact same sample of students.

  • Example 2 (Product Durability): A company testing tyre durability could give one person two different sets of tyres (new and old designs) to test, rather than using two different groups of drivers.

• Choosing Between Designs:

  • Reducing Variability: Matched pairs are often superior because they control for individual variability (e.g., human factors in driving or baseline intelligence in testing). This makes it easier to isolate the effect of the "treatment" or intervention.

  • Standard Error and Sample Size: Independent samples include naturally occurring variation between individuals. This extra variability increases the standard error ($se$), often requiring a larger sample size ($n$) to detect a statistically significant effect.

  • Efficiency: Matched pairs can use fewer participants because each participant provides two data points.

  • Practical Constraints: Matching may not always be possible or practical. It may be difficult to find perfect matches, or the experimental conditions themselves might prevent reuse of the same subjects (e.g., comparing populations of two different cities like Sydney and Melbourne must be independent).

Case Study 1: Financial Literacy and Population Means (Independent Samples)

• Context and Social Importance:

  • Australia has high global financial literacy rankings, yet there is widespread literacy variance within the country.

  • Financial literacy correlates with wealth accumulation, retirement planning, superannuation savings, economic empowerment for women, and protection against domestic violence.

  • The Claim: Overseas-born Australians (specifically from certain Asian countries) may have lower financial literacy rates compared to Australian-born residents.

• Study Data (Independent Samples):

  • Population 1 ($X_1$): Australian-born individuals ($n_1 = 219$).

  • Population 2 ($X_2$): Individuals born in four Asian countries: China, India, Sri Lanka, and Vietnam ($n_2 = 468$).

  • Testing Categories:

    • fl_basic: Scores for five basic financial literacy questions.

    • fl_stock: Scores for five stock market questions.

    • fl_super: Scores for two retirement/superannuation questions.

    • fl_sum: Total scores out of twelve.

• Defining Parameters and Hypotheses for Means:

  • $\mu_1$: Population mean score for Australian-born individuals.

  • $\mu_2$: Population mean score for overseas-born individuals.

  • Null Hypothesis ($H_0$): $\mu_1 = \mu_2$ or $\mu_1 - \mu_2 = 0$. (No difference in average scores).

  • Alternative Hypothesis ($H_A$): \mu_1 > \mu_2 or \mu_1 - \mu_2 > 0. (Australian-born have higher average scores - Upper tail test).

• Estimator and Standard Error (Independent):

  • Unbiased Estimator: $\bar{X}_1 - \bar{X}_2$.

  • Standard Error Calculation:         $se(\bar{X}_1 - \bar{X}_2) = \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}$

  • The estimator is consistent (approaches zero as sample sizes increase).

• The Test Statistic ($t$) and Distribution:

  • Formula:         $t = \frac{(\bar{X}_1 - \bar{X}_2) - (\mu_1 - \mu_2)}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}$

  • Distribution: Student-t distributed with degrees of freedom $df = n_1 + n_2 - 2$.

• Numerical Calculation and Decision:

  • Data: $n_1 = 219$, $n_2 = 468$, $\bar{x}_1 = 6.763$, $\bar{x}_2 = 6.596$, $s_1^2 = 8.971$, $s_2^2 = 10.417$.

  • Calculated Standard Error: $se(\bar{X}_1 - \bar{X}_2) = 0.2514$.

  • Calculated t-statistic: $t = \frac{6.763 - 6.596 - 0}{0.2514} = 0.6641$.

  • Degrees of Freedom: $219 + 468 - 2 = 685$.

  • Critical Value ($\alpha = 0.05$): For $df = 685$, $t_{0.05} \approx 1.645$ (using Excel: =T.INV(0.95, 685)).

  • Conclusion: Since t = 0.6641 < 1.645 and the $p$-value (0.2534) is greater than 0.05, we do not reject $H_0$. There is insufficient evidence to claim Australian-born individuals have higher financial literacy mean scores.

Case Study 2: Comparing Proportions (Independent Samples)

• Research Question: Is the proportion of women who pass the financial literacy test lower than the proportion of men?

  • Population 1 (Females): $\hat{p}_1$ is the sample pass proportion.

  • Population 2 (Males): $\hat{p}_2$ is the sample pass proportion.

• Defining Hypotheses for Proportions:

  • $H_0: p_1 = p_2$ (No difference between male and female proportions).

  • H_A: p_1 < p_2 (Lower tail test: female pass rate is lower than male pass rate).

• Sample Statistics:

  • Females: $n_1 = 344$, passes = 184, $\hat{p}_1 = \frac{184}{344} = 0.5349$.

  • Males: $n_2 = 343$, passes = 268, $\hat{p}_2 = \frac{268}{343} = 0.7813$.

• Standard Error and Test Statistic:

  • Variance Calculation:         $Var(\hat{p}_1) = \hat{p}_1(1 - \hat{p}_1) = 0.5349 \times (1 - 0.5349) = 0.2488$         $Var(\hat{p}_2) = \hat{p}_2(1 - \hat{p}_2) = 0.7813 \times (1 - 0.7813) = 0.1708$

  • Standard Error Formula:         $se(\hat{p}_1 - \hat{p}_2) = \sqrt{\frac{Var(\hat{p}_1)}{n_1} + \frac{Var(\hat{p}_2)}{n_2}} = 0.0349$

  • Test Statistic Formula:         $t = \frac{(\hat{p}_1 - \hat{p}_2) - (p_1 - p_2)}{se(\hat{p}_1 - \hat{p}_2)}$

  • Calculated Test Statistic:         $t = \frac{0.5349 - 0.7813 - 0}{0.0349} = -7.0505$

• Decision and Interpretation:

  • Critical Value ($\alpha = 0.05$): $-1.645$.

  • Results: Since t = -7.0505 < -1.645 and the $p$-value is $0.0000$, we reject $H_0$.

  • At the 5% significance level, there is strong evidence (also valid at 1%) that the proportion of females passing the test is lower than that of males.

Case Study 3: Matched Pairs Design (Sales Strategy Pilot)

• Background Scenario:

  • A multinational company facing stagnant sales implements a new strategy.

  • 50 sales employees ($n = 50$) are selected for a pilot program.

  • Performance data is collected for these specific individuals before and after training.

• Defining the Variables and Parameters for Differences:

  • $X_1$: Sales before training.

  • $X_2$: Sales after training.

  • Difference Variable ($D$): $D = X_2 - X_1$.

  • $\mu_D$: The mean of the differences in the population ($\mu_2 - \mu_1$).

  • Sample Mean of Differences ($\bar{D}$): $\bar{D} = \frac{1}{n} \sum_{i=1}^{n} D_i = \bar{X}_2 - \bar{X}_1$.

• Hypotheses for Matched Pairs:

  • $H_0: \mu_D = 0$ (Training had no effect on average sales).

  • H_A: \mu_D > 0 (Upper tail test: Training increased average sales).

• The Matched Pairs Test Statistic:

  • Standard Error Formula:         $se(\bar{D}) = \frac{s_D}{\sqrt{n}}$

  • Test Statistic Formula:         $t = \frac{\bar{D} - \mu_D}{s_D/\sqrt{n}}$

  • Distribution: Student-t distributed with $df = n - 1$ (where $n$ is the number of pairs).

• Numerical Calculation and Decision:

  • Data Highlights: $n = 50$, $\bar{D} = 147.6$, $s_D = 276.10$.

  • Calculated Standard Error: $se(\bar{D}) = \frac{276.10}{\sqrt{50}} = 39.05$.

  • Calculated t-statistic: $t = \frac{147.6 - 0}{39.05} = 3.78$.

  • Critical Value ($\alpha = 0.05, df = 49$): $1.676$ (using Excel: =T.INV(0.95, 49)).

  • Results: Since t = 3.78 > 1.676 and the $p$-value is $0.0002$, we reject $H_0$.

  • There is sufficient evidence that average sales increased post-training, justifying a full rollout of the strategy.

Distribution Properties and Tables

• T-Distribution vs. Z-Distribution:

  • As degrees of freedom ($df$) increase, the t-distribution approaches the standard normal (Z) distribution.

  • For large samples, critical values converge. For example, for a 0.05 upper tail test ($\alpha = 0.05$), the Z-critical value is $1.645$, and for $df = 685$, the t-critical value is very close at $1.647$.

• Excel Functions for Hypothesis Testing:

  • Critical Values: =T.INV(probability, df) (for lower tail) or =T.INV(1-alpha, df) (for upper tail).

  • P-values (Lower Tail): =T.DIST(t_statistic, df, TRUE).

  • P-values (Upper Tail): =1 - T.DIST(t_statistic, df, TRUE).

Key Terminology Summary

• Independent Samples: Samples drawn from two populations with no connection between individuals in Sample A and Sample B.

• Match Pairs (Paired Samples): Samples where each observation in one group is matched with a specific observation in the second group (often the same person before and after).

• Lower-tail/Upper-tail/Two-tail Tests: Refers to the region of rejection based on the alternative hypothesis ($H_A$).

• Consistency: A property of an estimator where its standard error approaches zero as the sample size increases.