Distribution of Sample Mean & Central Limit Theorem

Distribution of the Sample Mean

  • Why sample means vary
    • Each sample is only a subset → different observed values → different sample means (\bar{x}).
    • Leads to three guiding questions:
    1. What is the central/typical value of (\bar{x})?
    2. What is the variability (spread) of (\bar{x})?
    3. Does this variability follow a recognizable distributional pattern?
  • Symmetry expectation
    • For large n the distribution of (\bar{x}) is roughly symmetric → (\bar{x}) falls below and above \mu about 50 % of the time.
    • For small n symmetry may fail, but most (\bar{x}) values still cluster near \mu.
  • Theoretical guarantee: \mu_{\bar{x}} = \mu (mean of all sample means equals the population mean).

Unbiasedness & the Bull’s-Eye Metaphor

  • Unbiased estimator: Expected value of estimator equals the parameter.
    • E[\bar{x}] = \mu → \bar{x} is unbiased for \mu.
    • Other unbiased estimators mentioned:
    • Sample proportion \hat{p} for population proportion p.
    • Sample variance s^2 for population variance \sigma^2.
  • Bull’s-eye visual
    • Target center = true parameter.
    • Four scenarios:
    1. Wide scatter but centered → unbiased, high variance.
    2. Tight cluster, centered → unbiased, low variance (ideal).
    3. Wide scatter, shifted center → biased, high variance.
    4. Tight cluster, shifted center → biased, low variance.

Variability of the Sample Mean (Standard Error)

  • Infinite population (or sampling with replacement):
    • \sigma_{\bar{x}} = \dfrac{\sigma}{\sqrt{n}}
  • Finite population (sampling without replacement):
    • \sigma_{\bar{x}} = \sqrt{\dfrac{N - n}{N - 1}} \times \dfrac{\sigma}{\sqrt{n}}
    • N = population size, n = sample size.
  • Key implication: Increasing n ↓ \sigma_{\bar{x}} → estimates become more precise.

Numerical Visualization Example

  • Population parameters: \mu = 43\,660, \sigma = 2\,500.
  • Three normal curves for \bar{x}:
    • n = 25 → widest, flattest curve (largest standard error).
    • n = 100 → intermediate width/height.
    • n = 200 → tallest, narrowest curve (smallest standard error).

Desirable Properties of \bar{x}

  1. Unbiasedness: \mu_{\bar{x}} = \mu.
  2. Efficiency with n: Larger n ⇒ smaller \sigma_{\bar{x}}.

Central Limit Theorem (CLT)

  • Often called the fundamental theorem of statistics.
  • Statement (for "sufficiently large" n \ge 30):
    1. Distribution of \bar{x} is approximately normal, regardless of the population’s shape.
    2. Center: \mu_{\bar{x}} = \mu.
    3. Spread: \sigma_{\bar{x}} = \dfrac{\sigma}{\sqrt{n}}.
  • Special case: If the population itself is normal, all three properties hold for any n (even n < 30).

Visual Demonstrations of the CLT

  • Four population shapes examined:
    1. Bimodal (two humps).
    2. Uniform (all values equally likely).
    3. Exponential (right-skewed, higher likelihood near zero).
    4. Normal.
  • In every scenario, the sampling distribution of \bar{x} becomes (approximately) normal as n \to 30 or larger.
    • For the inherently normal population, normality of \bar{x} holds even at small n.

Error of Estimation (Sampling Error)

  • Defined as difference between a sample statistic and the corresponding population parameter.
    • For means: \text{Error} = \bar{x} - \mu.
    • Also called sampling error or estimation error.
  • Arises purely because we observe a sample rather than the full population.
  • Can be reduced by:
    • Increasing n (reduces standard error).
    • Using better sampling designs to avoid bias.

These notes cover the distribution, properties, and practical implications of the sample mean, capped by the Central Limit Theorem’s assurance that large-sample means behave normally and predictably.