Central Limit Theorem and Confidence Intervals of Sample Means

Introduction

• The previous lecture focused on quantifying the variability of the sample mean $\bar{X}$ as an estimator of the population mean $\mu$ using the standard error.

• The current lecture will use the Central Limit Theorem (CLT) to determine the sampling distribution for $\bar{X}$, independent of the population distribution from which the sample originates.

• The objective is to use the standard error and the sampling distribution to make inferences about $\mu$, particularly to establish confidence intervals around $\bar{X}$, which estimate a range of values likely containing $\mu$ (interval estimate).

Population and Sampling Distribution

• Population distribution: $X \sim N(\mu, \sigma)$

• The sampling distribution for $\bar{X}$, with a fixed sample size $n$, is centered at $\mu$ with a standard deviation of $\frac{\sigma}{\sqrt{n}}$.

• The sampling distribution results from taking multiple random samples of the fixed size $n$ and computing $\bar{X}$ for each sample.

• Key Point: The observed sample mean $\bar{X}$ is a random quantity; it is one observation from the sampling distribution for $\bar{X}$.

Properties of the Sampling Distribution of $\bar{X}$

• Mean of Sampling Distribution: The mean of the sampling distribution of $\bar{X}$ equals the population mean $\mu$.

• Standard Deviation of Sampling Distribution: The standard deviation of the sampling distribution of $\bar{X}$ is $\frac{\sigma}{\sqrt{n}}$, where $\sigma$ is the population standard deviation and $n$ is the sample size (this is termed as the "standard error").

• Understanding the probability distribution or model for the sampling distribution is essential, as it allows for the computation of probabilities regarding specific values for the sample mean.

• Central Limit Theorem (CLT): Remarkably, for nearly any distribution, the sampling distribution of $\bar{X}$ is approximately Normal if $n$ is sufficiently large.

Visual Representation of Sampling Distribution

• Sampling Distribution for $\bar{X}$: Two cases are demonstrated:

  • n = 10: The sampling distribution presents deviations due to its smaller size.

  • n = 100: The distribution is noticeably smoother and more consolidated as $n$ increases, shown with a population distribution that is Normal.

• Other distributions examined include:

  • Uniform Distribution: If data arises from a Normal distribution, the sampling distribution of $\bar{X}$ is precisely Normal.

  • Highly Skewed Distribution: A unimodal but highly skewed population distribution still exhibits characteristics of sampling distributions for small $n$ but approaches Normality as $n$ increases.

Formal Definition of the Central Limit Theorem

• The Central Limit Theorem states that given a random sample of $n$ observations, the sampling distribution of the sample mean $\bar{X}$ is approximately Normal, irrespective of the population distribution if $n$ is sufficiently large.

• The term "sufficiently large" is contingent upon the original population distribution:

  • The more similar the population distribution is to Normal (characteristics: symmetric, unimodal, no outliers), the smaller the $n$ that is necessary.

  • Conversely, samples taken from highly skewed distributions necessitate larger $n$.

Central Limit Theorem in Mathematical Form

• Let $X_1, X_2, \dots, X_n$ constitute a random sample from a distribution with mean $\mu$ and standard deviation $\sigma$. If $n$ is sufficiently large, the sampling distribution for the sample mean $\bar{X}$ can be approximated by $N(\mu, \frac{\sigma}{\sqrt{n}})$, a Normal distribution with mean $\mu$ and standard deviation $\frac{\sigma}{\sqrt{n}}$.

• Rule of Thumb: $n \geq 30$ is generally regarded as sufficient; however, larger $n$ might be mandated based on the level of skewness present in the population distribution.

Application of the Central Limit Theorem Example: BRFSS Data Analysis

• Consider the Behavioral Risk Factor Surveillance System (BRFSS) data, we explore the approximate sampling distribution of each sample mean:

  • Mean Age: 45.1, Standard Deviation: 17.2, Standard Error: 0.122

  • Mean Height: 67.2, Standard Deviation: 4.1, Standard Error: 0.029

  • Mean Weight: 169.7, Standard Deviation: 40, Standard Error: 0.283

Example: Computing Probabilities

• Given the true population mean age $\mu_{\text{age}} = 45$ years: What is the probability that $\bar{X}_{\text{age}}$ (observed sample mean) is greater than or equal to 45.1?

• According to CLT, $\bar{X}$ follows approximately $N(\mu_{\text{age}}=45, 0.122)$.

• Computation Steps to Determine Probability:

  • The standardized form can be calculated as follows: $P(\bar{X} \geq 45.1) = P(Z \geq \frac{45.1 - 45}{0.122})$

  • After calculating the Z-score, utilize statistical tables or computational software to find the associated probability.

Goals of Statistical Inference

• Upon acquiring data and calculating summary statistics, we proceed to statistical inference, which encompasses two prevalent types:

  • Hypothesis Testing: The purpose here is about making decisions using point estimators and their standard errors to evaluate alternate models of the underlying truth.

  • Point and Interval Estimation: The aim is to derive point estimates of population parameters alongside corresponding confidence intervals that likely contain the parameter value.

Motivation for Confidence Intervals

• The point estimate functions as our most accurate guess of the population parameter.

• Conversely, the confidence interval broadens the estimate to specify a range of probable values for the population parameter.

• Analogy: The point estimator acts like fishing with a spear (precise but narrow), while the confidence interval likens to fishing with a net (wider capturing range).

General Form for Confidence Intervals

• The point estimator $\bar{x}$ serves as the midpoint of the interval, with the confidence interval for a population mean expressed as: $\bar{x} \pm m$, where $m$ is the measure of the error margin for the point estimate.

• Selection of $m$ can be performed as follows:

  • The standard error for $\bar{x}$ is given by $\frac{\sigma}{\sqrt{n}}$.

  • By CLT, the sampling distribution for $\bar{x}$ is approximately $N(\mu, \frac{\sigma}{\sqrt{n}})$.

Properties of Normal Distribution for Confidence Functions

• The properties of the Normal distribution indicate that 95% of the distribution is encapsulated within 1.96 standard deviation units of the mean.

• For creating a confidence interval to include 95% of realizations of $\bar{x}$, the formulation becomes:
  $\bar{x} \pm 1.96 \times \frac{\sigma}{\sqrt{n}}$

• Explicitly, the margin of error $m$ centers around: $m = 1.96 \times \frac{\sigma}{\sqrt{n}}$.

Confidence Interval for Population Mean $\mu$

• A procedural framework is presented to create an interval for $\mu$ that adheres to the aforementioned probabilities.

• Due to the CLT, 95% of $\bar{x}$ realizations reside within 1.96 standard errors of $\mu$:
  \bar{x} - \mu < 1.96 \times \text{Standard Error}

• Thus, intervals constructed in the form $\bar{x} \pm 1.96 \times \text{Standard Error}$ will encapsulate the population mean $\mu$ for 95% of sampled means $\bar{x}$.

• The margin of error is established as $m = 1.96 \times \frac{\sigma}{\sqrt{n}}$.

Example: Air Pollution in Ann Arbor

• Measurement Target: Concentrations of Fine Particulate Matter (PM2.5) in Ann Arbor, with 25 measurements:

  • Variable Statistics:

  • Variable: PM2.5

  • Sample size ($n$): 25

  • Mean: 28.4

  • Standard Deviation: 4.72

  • Standard Error: 0.944

• Evaluating EPA Standards initiated by the Environmental Protection Agency (EPA) for a daily average of $\leq 35$ micrograms per cubic meter:

  • The aim is to ascertain whether Ann Arbor meets this EPA standard through the mean PM2.5 concentration to be less than or equal to 35 µg/m³.

Computing Confidence Intervals for Air Pollution

• To compute the 95% confidence interval for the mean PM2.5 in Ann Arbor:
  $\text{Confidence Interval} = \bar{x} \pm 1.96 \times \frac{4.72}{\sqrt{25}}$

• Result: For the mean of $28.4 \pm 1.96 \times 0.944$, which yields approximately from (26.55 to 30.25) or $(26.55, 30.25)$. Thus, confirming that the mean PM2.5 is below the EPA standard of 35.

Adjusting the Confidence Level

• Variations in confidence levels are feasible by selecting alternative $z$-values for the margin of error:

  • For instance, choosing $z = 2.58$ leads to a 99% confidence interval, framed as:
    $CI = \bar{x} \pm 2.58 \times \text{Standard Error}$

• Conceptually, escalating the confidence level results in a broader confidence interval to ensure capturing the true mean, implicating a trade-off between confidence and precision.

Example of Confidence Interval Adjustments: Air Pollution Case

• With the PM2.5 example at $n = 25$ where previously calculated as $28.4 \mu g/m^3$, to derive the 99% confidence intervals, compute:

• For 99% confidence interval: $28.4 \pm 2.58 \times (\frac{4.72}{\sqrt{25}})$ leading to bounds from approximately $(25.96, 30.84)$ — reaffirming below the EPA standard of 35.

General Formula for Margin of Error

• The margin of error for a confidence level $C$ is mathematically articulated as:
  $m = z^* \times \frac{\sigma}{\sqrt{n}}$, where $z^*$ represents the critical value encompassing the area $C$ in the standard Normal distribution.

Strategies for Narrowing Confidence Intervals:

• It is possible to refine a confidence interval by either:

  1. Reducing population variability $\sigma$ (often not attainable),

  2. Lowering the confidence level $C$ (and thus reducing certainty),

  3. Increasing the sample size $n$.

Thought Experiments: Comparing Confidence Intervals:

• Consider two measurement sets (X, Y) from one population with standard deviation $\sigma = 10$. Identify narrower confidence intervals:

  1. A 95% CI for $\bar{X}$ from $n = 10$ vs a 95% CI for $\bar{Y}$ from $n = 20$.

  2. A 95% CI for $\bar{X}$ from $n = 10$ vs a 99% CI for $\bar{Y}$ from $n = 10$.

  3. A 95% CI for $\bar{X}$ from $n = 10$ vs a 99% CI for $\bar{Y}$ from $n = 20$.

Sample Size Calculations

• Accurate sample size estimations are crucial for achieving a desired precision level in the point estimator.

  • The formula for the requisite sample size $n$ to maintain a confidence interval with a specified margin of error $m$ for a population mean is:
    $n = \left(\frac{z^* \sigma}{m}\right)^2$

Sample Size Example: Air Pollution

• In the FPM instance with a sample size $n = 25$ that yields a margin of error $m = 1.96 \times \frac{4.72}{\sqrt{25}} = 1.85 \mu g/m^3$.

• If aiming for a reduced margin of error of $m = 1 \mu g/m^3$, compute:
  $n = \left(\frac{1.96 \times 4.72}{1}\right)^2 = 85.58$. Rounded to $n = 86$ samples are needed.

Sample Size Calculation for 99% Confidence Interval

• For achieving a margin of error of $m = 1 \mu g/m^3$ using a 99% confidence level:
  $n = \left(\frac{2.58 \times 4.72}{1}\right)^2 = 148.29$. Hence at least $n = 149$ samples should be collected for the adjusted margin.

Summary and Key Ideas

• The Central Limit Theorem (CLT) explicates the sampling distribution for the sample mean (or sample proportion as it is a form of mean).

• Provided that the sample size $n$ is sufficiently large, $\bar{x}$ is approximately normally distributed regardless of the original measurement source.

• The utility of having a defined sampling distribution underpins the ability to compute probabilities related to $\bar{x}$ and serves vital for statistical inference processes, including both confidence intervals and hypothesis testing.