Notes on Confidence Intervals for the Mean

Confidence intervals for the population mean

• Goal: estimate μ, the mean of the population, using a sample. Attach a confidence level to quantify uncertainty about the interval around a point estimate.
• Key pieces:
  • Point estimate: the sample mean, \\bar{X}.
  • Margin of error (ME): the radius of the interval around the point estimate.
  • Confidence level (1 - α): the long-run proportion of intervals that would contain μ if we repeated the study many times.
  • Endpoints are determined by the sampling distribution of the estimator (\bar{X}) and the chosen critical value from a normal or t distribution.

Known population standard deviation σ: constructing a CI for μ

• Assumptions:

  • Independent identically distributed samples from a population with true mean μ and known standard deviation σ.
  • By the central limit theorem, the sampling distribution of the sample mean is approximately Normal for large n (exactly Normal if the population is Normal).

• Sampling distribution:

  • $\bar{X} \\sim \mathcal{N}\\big( \mu, \frac{\sigma^2}{n} \big).$

• Confidence interval for μ:

  • $\ ext{CI}<em>{\mu} = \bar{X} \pm z</em>{\alpha/2} \,\frac{\sigma}{\sqrt{n}}.$
  • Lower bound: $\bar{X} - z_{\alpha/2} \,\frac{\sigma}{\sqrt{n}}.$
  • Upper bound: $\bar{X} + z_{\alpha/2} \,\frac{\sigma}{\sqrt{n}}.$

• Margin of error (ME):

  • $\text{ME} = z_{\alpha/2} \,\frac{\sigma}{\sqrt{n}}.$
  • The width of the interval is $2\,\text{ME} = 2 \; z_{\alpha/2} \,\frac{\sigma}{\sqrt{n}}.$

• Confidence level and z-value relationship:

  • For a two-sided interval, α = 1 - (confidence level).
  • Common z-values:
  • 90% CI: $z<em>{\alpha/2} = z</em>{0.05} = 1.645.$
  • 95% CI: $z<em>{\alpha/2} = z</em>{0.025} = 1.96.$
  • 99% CI: $z<em>{\alpha/2} = z</em>{0.005} = 2.576.$
  • Higher confidence level ⇒ larger z_{\alpha/2} ⇒ wider interval (larger ME).

• Interpretation (frequentist):

  • If we repeated the sampling many times and computed a 95% CI each time, 95% of those intervals would contain μ.
  • A single computed interval either contains μ or it does not; the 95% refers to the long-run frequency of such intervals.

Unknown population standard deviation σ: using the sample standard deviation s

• When σ is unknown, replace σ with the sample standard deviation s, and use the t-distribution with n - 1 degrees of freedom.

• Confidence interval for μ:

  • $\text{CI}<em>{\mu} = \bar{X} \pm t</em>{\alpha/2, \; n-1} \,\frac{s}{\sqrt{n}}.$
  • Lower bound: $\bar{X} - t_{\alpha/2, \; n-1} \,\frac{s}{\sqrt{n}}.$
  • Upper bound: $\bar{X} + t_{\alpha/2, \; n-1} \,\frac{s}{\sqrt{n}}.$

• Distribution of the t-statistic:

  • $\frac{\bar{X} - \mu}{s/\sqrt{n}} \\sim \text{t}_{n-1}.$
  • As n grows large, the t-distribution converges to the standard normal distribution, so the z-interval becomes a good approximation when n is large.

• Relationship to z-intervals:

  • If σ is unknown and n is large, the t-critical value approximates the z-critical value, and the interval width becomes similar to the known-σ case.

• Margin of error with unknown σ:

  • $\text{ME} = t_{\alpha/2, \; n-1} \,\frac{s}{\sqrt{n}}.$

Relationship among confidence level, margin of error, and sample size

• With a fixed n and σ (known): increasing confidence level increases z_{\alpha/2} and thus increases ME, producing a wider interval.
• With a fixed desired margin of error E and a known σ, required sample size is approximately:
  • n \\approx \left( \\frac{z_{\alpha/2} \, \sigma}{E} \\right)^2.
• With unknown σ, replace σ with s and use the t-quantile:
  • n \\approx \left( \\frac{t_{\alpha/2, \; n-1} \, \; s}{E} \\right)^2, which is implicit in n and often solved iteratively.
• The standard error (spread) of the sample mean:
  • Known σ: $\text{SE}(\bar{X}) = \frac{\sigma}{\sqrt{n}}.$
  • Unknown σ: $\text{SE}(\bar{X}) = \frac{s}{\sqrt{n}}.$
• Sample size vs. margin of error relationship:
  • Margin of error decreases with larger n as $\text{ME} \\propto \\frac{1}{\sqrt{n}}.$
  • To achieve a smaller margin of error without changing confidence level or σ, collect more data.

Illustrative example related to sampling and coverage (conceptual)

• Suppose the true mean μ is known in a simulation (e.g., μ = 44 for a toy example) and we repeatedly draw samples of size n and compute CIs.

• In a simulation with 20 repeats:

  • Compute each sample mean x̄_i and the corresponding 95% CI.
  • Count how many intervals cover the true μ across the repeats.
  • If the true μ is 44 and 19 of the 20 CIs contain 44, that run demonstrates a 95% confidence-level behavior (about 5% of intervals may fail to cover in a finite simulation).

• Interpretation: over many repetitions, about 95% of CI's constructed with this method would cover the true μ, although any single interval may or may not contain μ.

• Structure of a confidence interval in this context:

  • Center (point estimate): x̄, the sample mean.
  • Margin of error: the distance from x̄ to either endpoint (ME).
  • The interval endpoints are: x̄ ± ME.

• Example interpretation for a real data set (given values are placeholders for illustration):

  • Suppose a study reports a sample mean of \bar{X} = 7.0 pounds with n = 144 and a known σ (or large-sample z-approximation) and a 95% CI.
  • The 95% CI would be: $[\bar{X} - z<em>{0.025} \,\frac{\sigma}{\sqrt{n}}, \\bar{X} + z</em>{0.025} \,\frac{\sigma}{\sqrt{n}} ].$
  • If σ is known to be, for example, 0.5 pounds, then the ME would be $ME = 1.96 \, \frac{0.5}{\sqrt{144}} = 1.96 \, \frac{0.5}{12} \approx 0.082.$
  • The CI would be approximately $[7.0 - 0.082, \; 7.0 + 0.082] = [6.918, 7.082].$

Quick reference: essential formulas to memorize

• Confidence interval for μ with known σ:

  • $\text{CI}<em>{\mu} = \bar{X} \pm z</em>{\alpha/2} \,\frac{\sigma}{\sqrt{n}}.$
  • $\text{ME} = z_{\alpha/2} \,\frac{\sigma}{\sqrt{n}}.$
  • $\bar{X} \\sim \mathcal{N}(\mu, \frac{\sigma^2}{n}).$

• Confidence interval for μ with unknown σ:

  • $\text{CI}<em>{\mu} = \bar{X} \pm t</em>{\alpha/2, \; n-1} \,\frac{s}{\sqrt{n}}.$
  • $\frac{\bar{X} - \mu}{s/\sqrt{n}} \\sim \text{t}_{n-1}.$

• Relationship between confidence level, z, and ME:

  • z_{\alpha/2} ext{ grows as the confidence level increases (e.g., 90%, 95%, 99%).}
  • $\text{ME} = z_{\alpha/2} \,\frac{\sigma}{\sqrt{n}}.$

• Margin of error and interval width:

  • $\text{Width} = 2 \cdot \text{ME} = 2 \, z_{\alpha/2} \,\frac{\sigma}{\sqrt{n}}.$

• Sample size for a desired margin E (known σ):

  • n \\approx \left( \\frac{z_{\alpha/2} \, \sigma}{E} \\right)^2.

• Sample size with unknown σ (using s and t):

  • n \\approx \left( \\frac{t_{\alpha/2, \; n-1} \, \; s}{E} \\right)^2,
  • note: this is implicit in n and often solved iteratively.

• Important practical note:

  • Always identify whether σ is known or unknown before choosing the z vs t framework.
  • The CLT justifies using the normal approximation for x̄ when n is reasonably large or when the population is normal.

• Real-world relevance:

  • Confidence intervals quantify uncertainty in estimation and guide decisions that depend on the range of plausible population means.
  • Understanding the trade-off between confidence level, margin of error, and sample size helps in experimental design and data collection planning.