Comparing Two Population Means

Outline

Two populations
Comparing Two Means
- Confidence Intervals
- Hypothesis Tests
An experiment
What happens if the variances aren’t equal?

Overview

Previously, the focus was on a 'reference' value and tests around it.
Example: Testing if the body weight of the US adult population has changed over time.
- Null Hypothesis ( $H_0 : µ = 177$ lbs)
- Alternative Hypothesis ( $H_A : µ \neq 177$ lbs)

Two Populations

Many questions involve comparing two different populations.
Notation: Xg,i is the $i^{th}$ unit in the $g^{th}$ group, where each group corresponds to a population with its own parameter $\mu_g$ .

Null Hypothesis Examples

Comparing body weight of US adult population (group 1) to Canadian adult population (group 2).
- $H0 : µ1 = µ_2$
Comparing life expectancy of emperor penguins ( $\muE$ ) to king penguins ( $\muK$ ).
- $H0 : µE \geq µ_K$
Comparing proportions for each candidate in an election.
- $H0 : p1 = p_2$

Alternative Hypothesis Examples

Comparing body weight of US adult population (group 1) to Canadian adult population (group 2) to see if average weights are different.
- $HA : µ1 \neq µ_2$
Comparing life expectancy of emperor penguins ( $\muE$ ) to king penguins ( $\muK$ ) to test if the life expectancy of king penguins is now longer than emperor penguins.
- HA : µE < µ_K
Comparing proportions for each candidate in an election to determine if the result of an election can be called.
- $HA : p1 \neq p_2$

Overview

Using CIs and hypothesis testing for comparing two population means or two population proportions.
- Examples: tensile strength of two procedures for producing steel, proportion of users that click on an ad for two variations on the graphics
Focusing on differences in means first.

Notation

Mean and variance of population g = 1, 2 are $\mug$ and $\sigma^2g$ .
Samples from each population:
- $X{11}, …, X{1n_1}$ from population 1
- $X{21}, …, X{2n_2}$ from population 2
Sample mean and sample variances:
- $\bar{X}g = \frac{1}{ng} \sum{i=1}^{ng} X_{g,i}$
- $S^2g = \frac{1}{ng - 1} \sum{i=1}^{ng} (X{g,i} - \bar{X}g)^2$

Estimating the Parameter

Estimating $\mu1 - \mu2$ using $\bar{X}1 - \bar{X}2$ .
Sampling distribution of $\bar{X}1 - \bar{X}2$ :
- $E[\bar{X}1 - \bar{X}2] =$
Variance:
- Assumptions: uncorrelated samples within each population (Var[ $\bar{X}1$ ] = , Var[ $\bar{X}2$ ] = )
If $n1, n2$ are large enough:
- $\bar{X}_1 \sim$
- $\bar{X}_2 \sim$
What about Var[ $\bar{X}1 - \bar{X}2$ ]?

The Sampling Distribution of $\bar{X}1 - \bar{X}2$

If the two sample means are uncorrelated, then
- $Var[\bar{X}1 - \bar{X}2] = \frac{\sigma^21}{n1} + \frac{\sigma^22}{n2}$
If $n1, n2 \geq 30$ , then
- $\bar{X}1 - \bar{X}2 \sim N(\mu1 - \mu2, \frac{\sigma^21}{n1} + \frac{\sigma^22}{n2})$

Variance Assumption

$\bar{X}1 - \bar{X}2 \sim N(\mu1 - \mu2, \frac{\sigma^21}{n1} + \frac{\sigma^22}{n2})$
Assumption: the two populations might have different population means but have the same variance ( $\sigma^21 = \sigma^22 = \sigma^2$ ).
- Then: $\frac{\sigma^21}{n1} + \frac{\sigma^22}{n2} = \sigma^2(\frac{1}{n1} + \frac{1}{n2})$
- And: $\bar{X}1 - \bar{X}2 \sim N(\mu1 - \mu2, \sigma^2(\frac{1}{n1} + \frac{1}{n2}))$

Estimating the Variance

Estimating the shared variance $\sigma^2$ by combining the observed sample variances $s^21$ and $s^22$ .
How to combine them into an estimate of $\sigma^2$ ?
- Could consider averaging them: $s^2 = \frac{s^21 + s^22}{2}$ . Potential issue: $n1$ may not equal $n2$ .

Pooled Estimate of the Variance

Weighted average:
- $s^2p = \frac{(n1 - 1)s^21 + (n2 - 1)s^22}{(n1 + n_2 - 2)}$ (pooled sample variance)
Example:
- If $n1 = 14$ and $n2 = 14$ , $s^21 = 10$ and $s^22 = 8$ , then what is $s^2_p$ ?
- If $n1 = 14$ and $n2 = 7$ , $s^21 = 10$ and $s^22 = 8$ , then what is $s^2_p$ ?
R function:
- pooledVar = function(sampVar1,sampVar2,n1,n2){ ((n1-1)*sampVar1 + (n2-1)*sampVar2)/(n1 + n2 - 2) }

WHAT IS THE APPROPRIATE DISTRIBUTION

What is the Appropriate Distribution

Recall: Form for a standardized random variable?
Recall: $\bar{X}1 - \bar{X}2 \sim N(\mu1 - \mu2, \sigma^2(\frac{1}{n1} + \frac{1}{n2}))$
Estimate $\sigma^2(\frac{1}{n1} + \frac{1}{n2})$ with $s^2p(\frac{1}{n1} + \frac{1}{n_2})$
- $\frac{(\bar{X}1 - \bar{X}2) - (\mu1 - \mu2)}{\sqrt{s^2p(\frac{1}{n1} + \frac{1}{n2})}} \sim t(n1 + n_2 - 2)$
- (Approximately holds if $n1, n2 \geq 30$ )
Punchline: Use a 't' distribution for confidence intervals/hypothesis tests for the difference of two means.

Confidence Intervals for $\mu1 - \mu2$

If:
- $\sigma^21 = \sigma^22$
- $\bar{X}1 - \bar{X}2$ is normal (primarily: $n1, n2 \geq 30$ )
Then a $(1 - \alpha)100\%$ CI for $\mu1 - \mu2$ is:
- $\bar{X}1 - \bar{X}2 \pm t{\alpha/2}(n1 + n2 - 2)\sqrt{s^2p(\frac{1}{n1} + \frac{1}{n2})}$

Example

Comparing two catalysts in terms of the mean yield of a chemical process.
- $n_1 = 32$ chemical processes with catalyst 1
- $n_2 = 35$ chemical processes with catalyst 2
Experimental results:
- Catalyst 1: $\bar{x}1 = 45.0313$ and $s1 = 3.1976$
- Catalyst 2: $\bar{x}2 = 50.4000$ and $s2 = 2.7246$
Construct a 97% CI for $\mu1 - \mu2$ .
- What assumptions are needed for the validity of the CI?
- Result: [-6.974882, -3.762618]
Another Example With Raw data:
- x1 = c(48, 43, 46, 40, 41, 46, 50, 41, 50, 42, 40, 44, 44, 49, 45, 49, 46, 48, 44, 44, 48, 48, 44, 44, 41, 49, 48, 40, 43, 42, 45, 49)
- x2 = c(54, 50, 48, 48, 54, 53, 51, 50, 53, 52, 53, 51, 52, 50, 54, 51, 47, 54, 50, 52, 46, 46, 50, 50, 45, 47, 47, 52, 50, 51, 50, 52, 51, 55, 45)
R code:
- t.test(x1,x2, var.equal=TRUE, conf.level = .97)
- 97 percent confidence interval: -6.974882 -3.762618

Hypothesis tests for $\mu1 - \mu2$

Follows the same overall structure as other tests.
Specify reference value $\Delta0$ and $\alpha$ (often $\Delta0 = 0$ ).
Tests:
- $H0 : µ1 - µ2 = \Delta0$ vs $HA : µ1 - µ2 \neq \Delta0$
- $H0 : µ1 - µ2 \geq \Delta0$ vs $HA : µ1 - µ2 < \Delta0$
- $H0 : µ1 - µ2 \leq \Delta0$ vs $HA : µ1 - µ2 > \Delta0$
Observed test statistic:
- $T{obs} = \frac{\bar{x}1 - \bar{x}2 - \Delta0}{\sqrt{s^2p(\frac{1}{n1} + \frac{1}{n_2})}}$
- Compared to the $t(n1 + n2 - 2)$ distribution.
Catalyst Example using a significance level $\alpha = 0.03$ :
- $T_{obs} = -7.4166$ , p-value = $3.173e-10$

An Experiment

Do students that sit in front have a different average height than the back?
Confidence Interval
Hypothesis Test

Key assumption behind pooling

$\sigma^21 = \sigma^22 = \sigma^2 \implies \frac{\sigma^21}{n1} + \frac{\sigma^22}{n2} = \sigma^2(\frac{1}{n1} + \frac{1}{n2})$
Estimate $\sigma^2$ with the pooled sample variance:
- $s^2p = \frac{(n1 - 1)s^21 + (n2 - 1)s^22}{(n1 + n_2 - 2)}$
What happens if we don’t make this assumption?

Unequal variances

Since $Var[\bar{X}1 - \bar{X}2] = \frac{\sigma^21}{n1} + \frac{\sigma^22}{n2}$ , estimate it by plugging in the sample variances directly:
- $\frac{s^21}{n1} + \frac{s^22}{n2}$
Observed test statistic:
- $T{obs} = \frac{\bar{x}1 - \bar{x}2 - \Delta0}{\sqrt{\frac{s^21}{n1} + \frac{s^22}{n2}}}$

Welch’s test: Unequal variances

Equal variances: the correct distribution is $t(n1 + n2 - 2)$
Unequal variances: the correct distribution is still a t, but the degrees of freedom is harder to calculate:
- $df = \frac{(\frac{s^21}{n1} + \frac{s^22}{n2})^2}{\frac{(\frac{s^21}{n1})^2}{n1 - 1} + \frac{(\frac{s^22}{n2})^2}{n2 - 1}}$
R syntax:
- t.test(x1,x2, var.equal=TRUE) (Equal variances)
- t.test(x1,x2) (Unequal variances)
Punchline: Just omit the ‘var.equal = TRUE’ argument

Comparing Two Population Means

Comparing Two Population Means

Outline

Overview

Two Populations

Null Hypothesis Examples

Alternative Hypothesis Examples

Overview

Notation

Estimating the Parameter

The Sampling Distribution of $\bar{X}<em>1 - \bar{X}</em>2$

Variance Assumption

Estimating the Variance

Pooled Estimate of the Variance

WHAT IS THE APPROPRIATE DISTRIBUTION

What is the Appropriate Distribution

Confidence Intervals for $\mu<em>1 - \mu</em>2$

Example

Hypothesis tests for $\mu<em>1 - \mu</em>2$

An Experiment

Key assumption behind pooling

Unequal variances

Welch’s test: Unequal variances

Comparing Two Population Means

Comparing Two Population Means

Outline

Overview

Two Populations

Null Hypothesis Examples

Alternative Hypothesis Examples

Overview

Notation

Estimating the Parameter

The Sampling Distribution of Xˉ<em>1−Xˉ</em>2\bar{X}<em>1 - \bar{X}</em>2Xˉ<em>1−Xˉ</em>2

Variance Assumption

Estimating the Variance

Pooled Estimate of the Variance

WHAT IS THE APPROPRIATE DISTRIBUTION

What is the Appropriate Distribution

Confidence Intervals for μ<em>1−μ</em>2\mu<em>1 - \mu</em>2μ<em>1−μ</em>2

Example

Hypothesis tests for μ<em>1−μ</em>2\mu<em>1 - \mu</em>2μ<em>1−μ</em>2

An Experiment

Key assumption behind pooling

Unequal variances

Welch’s test: Unequal variances

The Sampling Distribution of $\bar{X}<em>1 - \bar{X}</em>2$

Confidence Intervals for $\mu<em>1 - \mu</em>2$

Hypothesis tests for $\mu<em>1 - \mu</em>2$