9/18 Notes on Normal Distribution, Chi-Square, and Inference (Transcript Summary)

Notation and Setup

• Let the population be iid observations: 𝑋ᵢ ~ N(𝜇, 𝜎²) for i = 1, …, n, with independence assumed.
• Sample mean:
  $\bar{X} = \frac{1}{n}\sum<em>{i=1}^n X</em>i$
• Sample variance (unbiased):
  $S^2 = \frac{1}{n-1}\sum<em>{i=1}^n (X</em>i - \bar{X})^2$
• Standardization to normal form:
  • For each observation,
    $Z<em>i = \frac{X</em>i - \mu}{\sigma} \sim N(0,1)$
  • Each $Z<em>i^2 \sim \chi^2</em>1$ and the sum $\sum<em>{i=1}^n Z</em>i^2 \sim \chi^2_n$
• The transcript emphasizes the independence of the Xᵢ’s and the idea that you can analyze one observation at a time or the average; the left-hand pieces can be united into a full distributional picture.
• Key intuition: dividing by the appropriate scale converts to standard normal or chi-square forms; combining independent pieces leads to chi-square sums and, with ratios, to Fisher distributions.

Standard Normal and Normalization

• Distribution of the sample mean:
  $\frac{\bar{X} - \mu}{\sigma/\sqrt{n}} \sim N(0,1)$
• Squared standard normal for the mean:
  $(\frac{\bar{X} - \mu}{\sigma/\sqrt{n}})^2 = \frac{n(\bar{X}-\mu)^2}{\sigma^2} \sim \chi^2_1$
• The derivation often uses the idea that the single standardized observation and the sample mean (properly scaled) follow standard normal or chi-square forms.
• The transcript also discusses treating the bottom denominator as a constant (the population sigma) when focusing on the distributional form of the standardization.
• Important distinction: Xᵢ are from the same population, so they share the same 𝜇 and 𝜎, which justifies the standardization and the use of χ² and t distributions in subsequent inferences.

Chi-Squared Distribution and Sample Variance

• Distribution of the sample variance (assuming normal population):
  $(n-1)\,S^2/\sigma^2 \sim \chi^2_{n-1}$
• Therefore, the sampling distribution of S² is:
  $S^2 \sim \sigma^2 \cdot \frac{\chi^2_{n-1}}{n-1}$
• Consequence: S² is independent of \bar{X} for normal data; both are sufficient statistics for (𝜇, 𝜎²) in the normal model.
• Degrees of freedom intuition: you lose one degree of freedom for estimating the mean when forming (Xᵢ - X̄)², hence the n-1 in the chi-square argument.
• The transcript links the components: the sum of squares contributed by the centered data has chi-square behavior with n-1 df, while the sum of squares of standardized single observations contributes to chi-square with n df when considering the entire set of Zᵢ² terms.

Confidence Intervals for Mean and Variance

• Mean confidence interval for μ when 𝜎² is known:
  $\bar{X} \pm z_{\alpha/2}\, \frac{\sigma}{\sqrt{n}}$
• Mean confidence interval for μ when 𝜎² is unknown (use t distribution):
  $\bar{X} \pm t_{n-1, \alpha/2}\, \frac{S}{\sqrt{n}}$
• Variance confidence interval (for 𝜎²):
  $\left( \frac{(n-1)S^2}{\chi^2<em>{1-\alpha/2, \, n-1}}, \; \frac{(n-1)S^2}{\chi^2</em>{\alpha/2, \, n-1}} \right)$
• Rationale for n-1 in the denominator of S²: estimating 𝜇 consumes one degree of freedom, reducing the degrees of freedom available for variance estimation.
• Connection to the standard normal and chi-square: the z-based interval uses a standard normal quantile; the t-based interval uses a t-distribution with df = n-1; the variance interval uses chi-square quantiles with df = n-1.
• The transcript emphasizes building intuition from simple “add/subtract” steps and then recognizing the resulting distributions (z, t, χ²).

Fisher Distribution and Variance Ratios

• If U ∼ χ²{d1} and V ∼ χ²{d2} are independent, then
  $F = \frac{U/d1}{V/d2} \sim F_{d1, d2}$
• Using two independent samples with known or unknown population variances:
  • For two samples with known population variances, Fisher distribution applies to the ratio of variances.
  • In practice, with sample variances Sx² and Sy² and unknown population variances σx², σy², we use the standard form
    $F = \frac{\frac{(n<em>x-1) S</em>x^2}{\sigma<em>x^2}}{\frac{(n</em>y-1) S<em>y^2}{\sigma</em>y^2}} \sim F<em>{n</em>x-1, \, n_y-1}.$
• A common simplified version (used in practice) states that, when both population variances are unknown, the ratio of two sample variances adjusted by their degrees of freedom follows an F distribution with df1 = nx - 1 and df2 = ny - 1.
• Real-world example from the transcript: comparing variance of height between two populations (e.g., US vs France) using the F distribution to test for equality of variances.
• Key takeaway: F-distribution arises from the ratio of two scaled χ² variables, each coming from an independent normal sample variance estimate.

Two-Sample Inference and Difference of Means

• For the difference in population means (μ₁ − μ₂): the standard estimators and intervals follow the same pattern as single-sample inference, with added structure for the variance terms.
• When variances are known, a two-sample z-based interval for the difference is:
  $(\bar{X}<em>1 - \bar{X}</em>2) \pm z<em>{\alpha/2} \sqrt{ \frac{\sigma</em>1^2}{n<em>1} + \frac{\sigma</em>2^2}{n_2} }$
• When variances are unknown but assumed equal, use the pooled variance S_p² and a t-distribution with df = n₁ + n₂ − 2 (the classic two-sample t-test).
• If variances are unknown and not assumed equal (Welch’s approach), use an approximate degrees of freedom (Welch–Satterthwaite) and a t-distribution with that df.
• The transcript reflects the idea of forming a “theta1, theta2” interval for the difference and shows that the same logic (normal or t-based with a standard error) applies; the distributional form of the endpoints follows from the same distributional algebra.
• In all cases, the variance terms inside the standard error come from the estimated variances (either known, pooled, or separate), and the critical value comes from z, t, or their pooled/approximate counterparts.

Hypothesis Testing, Critical Regions, and Conceptual Flow

• The lecture emphasizes that hypothesis testing is a structured application of the same distributional logic used for confidence intervals:
  • Define a test statistic based on the estimator (e.g., sample mean, sample variance, or a ratio like F).
  • Determine its distribution under the null hypothesis.
  • Decide a rejection region based on a chosen significance level α.
  • Use the observed statistic to accept/reject the null hypothesis.
• The instructor notes that there is nothing magical about the process; it is built from normal-based foundations, and the same math underpins both interval estimation and hypothesis testing.
• The design of experiments perspective is integrated: choose estimators, understand their sampling distributions, and plan hypothesis tests accordingly; data collection is guided by the questions you want to answer and the required power.

Practical Considerations: Data Work and Mini-Project

• Discussion of a “mini-project” workflow:
  • Propose a problem statement based on a real client or publicly available dataset.
  • Gather data (public sources allowed); perform preliminary analyses to identify potential hypotheses.
  • Propose a plan for analyses (e.g., regression, classification, or hypothesis tests) and justify data needs.
  • If the given data are insufficient to answer the question, justify collecting additional data or seeking external data sources.
  • Frame the deliverables as a proposal to a client: what problem you will solve, what analyses you will run, what model(s) you will test, and what data you will collect.
• The discussion includes practical considerations such as timelines, office hours, and the importance of clarifying the client problem and expected outputs.
• The speaker emphasizes that the same statistical logic applies across different estimators (mean, variance, proportions, etc.) and that understanding the underlying distributional reasoning helps interpret results in hypothesis testing and interval estimation.

Connections to Foundational Principles and Takeaways

• Core idea: The normal model leads to chi-square, t, and F distributions for derived statistics like S² and ratios of variances; these in turn underpin confidence intervals and hypothesis tests.
• Degrees of freedom (df) reflect the number of independent pieces of information available after estimating parameters (e.g., subtracting 1 df for estimating μ to form (Xᵢ − X̄)², yielding n−1 for the variance term).
• The same algebraic approach (add/subtract, normalize by appropriate scale) underpins both interval estimation and hypothesis testing, making these methods logically connected.
• When comparing two populations, Fisher’s F statistic and the related F distribution allow testing for equality of variances; when comparing means, z/t-based intervals and tests hinge on the respective standard errors derived from the data.
• Practical application: the framework guides data analysis pipelines from estimation to testing to experimental design; both the theory and the practice revolve around understanding the sampling distributions of estimators and test statistics.
• Real-world emphasis: the importance of clear problem formulation, data collection strategy, and justification for using particular estimators or distributions in hypothesis testing and interval estimation.