Chi-Square Distribution and the Goodness-of-Fit Test

Foundations of Categorical Data Analysis Beyond Two Proportions

• Transition from Single/Two-Proportion Analysis to Multi-Category Distributions:

  • Traditional inferential statistics focuses on problems involving either one proportion or two proportions.

  • When evaluating one or two proportions, inferential procedures rely on the normal distribution model and the standard normal test statistic ($z$-statistic):         $z = \frac{\text{Observed} - \text{Expected}}{\text{Standard Error}}$

  • When categorical variables involve more than two categories (three or more proportions, such as eye color classified as brown, black, blue, or yellow), the normal distribution model is no longer mathematically appropriate or valid.

  • A different probability distribution is required to evaluate multinomial categorical distributions: the chi-square distribution ($\chi^2$ distribution).

• Nomenclature and Notation:

  • The symbol is written as a Greek letter chi with an exponent of two: $\chi^2$.

  • Pronunciation is strictly "kai-square" (not "chee" or "chai").

  • When written with its defining parameter, it is expressed with the degrees of freedom in parentheses: $\chi^2(df)$.

• Primary Analytical Methods Using the Chi-Square Distribution:

  • Goodness-of-Fit Test: Evaluates whether an observed sample distribution across multiple categories matches a specified or hypothesized population distribution.

  • Test of Independence: Evaluates whether two categorical variables are statistically independent or associated.

Mathematical and Theoretical Properties of the Chi-Square Distribution

• Asymmetry and Curve Geometry:

  • Unlike the normal distribution, which is symmetric and bell-shaped, the chi-square distribution is not bell-shaped.

  • The chi-square distribution is strictly skewed to the right (positively skewed).

  • The distribution features a long tail extending infinitely to the right; the probability density decreases gradually as values increase along the positive horizontal axis.

• Domain and Range Restrictions:

  • Chi-square values are strictly non-negative: $\chi^2 \ge 0$.

  • A chi-square test statistic can never take on a negative value because it represents a sum of squared differences divided by strictly positive expected values.

• Parameters and Distributional Moments:

  • While the normal distribution is defined by two parameters—mean ($\mu$) and standard deviation ($\sigma$)—the chi-square distribution is completely governed by a single parameter: degrees of freedom ($df$).

  • Mean: The mean of a chi-square distribution equals its degrees of freedom:         $\text{Mean} = df$

  • Variance: The variance of a chi-square distribution equals twice its degrees of freedom:         $\text{Variance} = 2 \times df$

  • Standard Deviation: The standard deviation is the square root of the variance:         $\text{Standard Deviation} = \sqrt{2 \times df}$

  • Numerical Example: For a chi-square distribution with $df = 5$:

    • $\text{Mean} = 5$

    • $\text{Variance} = 2 \times 5 = 10$

    • $\text{Standard Deviation} = \sqrt{10}$

• P-Value Directionality:

  • In two-sided or one-sided normal tests, areas under the curve can be left-tailed, right-tailed, or split between two tails.

  • In a chi-square distribution, the area corresponding to the $p$-value is always an area in the right tail ($P(\chi^2 \ge \text{test statistic})$). Left-tail areas are never used to compute the $p$-value for goodness-of-fit or independence tests.

Technology Implementation: Computing Cumulative Probabilities with TI-84

• Calculator Navigation Path:

  • Press [2nd] followed by [VARS] to access the [DISTR] (Distributions) menu.

  • Scroll down and select $\chi^2\text{cdf}$.

• Input Syntax and Arguments:

  • Syntax: $\chi^2\text{cdf}(\text{lower bound}, \text{upper bound}, df)$.

  • To capture the entire right-tail area extending to positive infinity, use an arbitrarily large number such as $10^{10}$ for the upper bound.

• Crucial Distinction: CDF vs. PDF:

  • $\chi^2\text{cdf}$ computes the cumulative distribution function (the total probability area over a continuous interval).

  • $\chi^2\text{pdf}$ computes the probability density function (the height of the density curve at a single discrete point) and must never be used to calculate a $p$-value.

  • Multiple-choice examinations frequently include distractors calculated using $\chi^2\text{pdf}$ to penalize incorrect cursor selections.

• Exam Preparation Note on Visualizing Curves:

  • While students are not required to sketch chi-square curves on the examination, exam questions involving normal models explicitly require drawing the normal curve and accurately shading the target probability region. Preparation must begin well in advance rather than within $48\,\text{hours}$ of the test.

• Worked Numerical Examples:

  • Example 1: Find the area under a chi-square distribution with $df = 5$ for values greater than $5$:         $\text{Lower Bound} = 5$         $\text{Upper Bound} = 10^{10}$         $df = 5$         $\text{Calculator Command: } \chi^2\text{cdf}(5, 10^{10}, 5)$

  • Example 2: Find the area under a chi-square distribution with $df = 5$ for values greater than $10.3$:         $\text{Lower Bound} = 10.3$         $\text{Upper Bound} = 10^{10}$         $df = 5$         $\text{Calculator Command: } \chi^2\text{cdf}(10.3, 10^{10}, 5)$

Chi-Square Goodness-of-Fit Test Methodology

• Purpose:

  • To determine whether an observed frequency distribution of counts across $k$ categories differs significantly from a hypothesized or theoretical distribution.

• Hypothesis Formulation:

  • Null Hypothesis ($H_0$): The distribution of categorical counts in the population is the same as the hypothesized distribution. Any discrepancies between observed and expected counts are minor and attributable entirely to sampling variation ("due to chance").

  • Alternative Hypothesis ($H_a$): The distribution of categorical counts in the population is not the same as the hypothesized distribution (it is different).

  • Metaphor for Sampling Variation ("Due to Chance"): If an individual is asked to draw three identical circles, the resulting shapes will never be mathematically identical despite attempting to draw them identically. Minor discrepancies are inherent to natural processes and sampling variation. Discrepancies are considered due to chance when they fall within standard statistical tolerance.

• Conditions for Inference:

  1. Independence Condition: Data must be collected using a random sample or an independent sampling mechanism.

  2. Expected Count Condition: The expected frequency count for every single category must be at least five:         $E_i \ge 5 \quad \text{for all } i$

     • If any expected cell count is less than $5$, the chi-square goodness-of-fit test cannot be validly performed.

Step-by-Step Computational Procedure for the Goodness-of-Fit Test

• Step 1: Determine the Degrees of Freedom ($df$):

  • In a goodness-of-fit test, degrees of freedom depend on the number of categories ($k$), not the total individual sample size ($n$):         $df = k - 1$

  • Example: If studying eye color across $4$ categories (black, brown, blue, yellow), $df = 4 - 1 = 3$.

• Step 2: Calculate the Expected Counts ($E_i$):

  • For each category $i$, multiply the total sample size ($n$) by the hypothesized baseline proportion ($p_i$):         $E_i = n \times p_i$

  • Critical Warning: The total sample size ($n$) must be used in this calculation. Observed counts ($O_i$) must never be multiplied by the baseline proportions to obtain expected values.

• Step 3: Calculate Category Contributions and the Test Statistic:

  • For each category, compute its squared difference from the expected value divided by the expected value (termed the category's contribution to chi-square):         $\text{Contribution}_i = \frac{(O_i - E_i)^2}{E_i}$

  • Sum the contributions across all $k$ categories to obtain the overall chi-square test statistic:         $\chi^2 = \sum_{i=1}^{k} \frac{(O_i - E_i)^2}{E_i}$

• Step 4: Compute the P-Value:

  • Use the cumulative chi-square distribution with the computed test statistic as the lower bound:         $p\text{-value} = P(\chi^2 \ge \text{test statistic}) = \chi^2\text{cdf}(\text{test statistic}, 10^{10}, df)$

• Step 5: Statistical Decision and Conclusion:

  • If the $p$-value is large (e.g., $p > 0.10$), there is insufficient evidence to reject the null hypothesis. Conclude that the observed distribution is consistent with the hypothesized distribution.

  • If the $p$-value is small ($p \le \alpha$), reject the null hypothesis in favor of the alternative.

Comprehensive Case Study: Student Perceptions of ChatGPT Utility

• Study Context and Definitions:

  • ChatGPT is defined as an open-access online chatbot that allows users to ask questions and request content.

  • The study examines whether the distribution of student opinions regarding the usefulness of ChatGPT in an introductory statistics course (STT 200) changed between 2023 and a later semester (lecture section 2).

• Baseline Claimed Distribution (Fall 2023):

  • Extremely Useful: $26\%$

  • Very Useful: $35\%$

  • Somewhat Useful, Not Useful, Not at All Useful: remaining categorical baseline percentages totaling $100\%$.

• Sample Data and Observed Counts ($n = 214$):

  • Total sample size analyzed: $n = 214$ students.

  • Total categories evaluated: $k = 5$.

  • Degrees of freedom: $df = 5 - 1 = 4$.

  • Observed counts ($O_i$):

    • Extremely Useful: $O_1 = 56$

    • Very Useful: $O_2 = 72$

    • Somewhat Useful: $O_3 = 68$

    • Not Useful & Not at All Useful: remaining observed counts from the cohort.

• Expected Counts Calculation ($E_i = n \times p_i$):

  • Extremely Useful: $E_1 = 214 \times 0.26 = 55.64$

  • Very Useful: $E_2 = 214 \times 0.35 = 74.90$

  • Somewhat Useful: $E_3 = 62.00$

  • All expected counts exceed $5$ ($E_i \ge 5$), satisfying the condition for chi-square inference.

• Contribution Calculations:

  • Category 1 (Extremely Useful):         $\text{Contribution}_1 = \frac{(56 - 55.64)^2}{55.64}$

  • Category 2 (Very Useful):         $\text{Contribution}_2 = \frac{(72 - 74.90)^2}{74.90}$

  • Category contributions are calculated analogously for all five levels and summed to produce the overall $\chi^2$ test statistic.

• Hypothesis Statements:

  • $H_0$: The distribution of opinions about ChatGPT among STT 200 students is the same as the 2023 distribution.

  • $H_a$: The distribution of opinions about ChatGPT among STT 200 students is not the same as the 2023 distribution.

• Inference Results and Practical Interpretation:

  • The test yields a $p$-value greater than $0.10$ ($p > 0.10$).

  • Because the $p$-value is greater than $0.10$, there is very little evidence against the null hypothesis.

  • Conclusion: Fail to reject $H_0$. The evidence indicates that student opinions regarding the usefulness of ChatGPT have remained stable and are the same as the distribution observed in 2023.

Questions & Classroom Discussion

• Top Hat Problem: Area Calculation on a Chi-Square Distribution:

  • Question: What is the area under the chi-square distribution with degrees of freedom $df = 4$ that is greater than $10.1$?

  • Setup and Input: $\chi^2\text{cdf}(10.1, 10^{10}, 4)$.

  • Result: $0.0388$.

  • Student Dialogue and Troubleshooting: Several students initially obtained an incorrect result or had their submissions flagged as wrong before entering the verified calculation of $0.0388$.

• Live Class Polling vs. Archival Cohort Data:

  • A live poll asked students: "How useful do you consider ChatGPT to be?"

  • Live Poll Results: Extremely useful = $108$, Very useful = $180$, Somewhat useful = $60$, Not useful = $8$, Not at all useful = $2$.

  • Discrepancy and Resolution: Although the classroom attendance was approximately $254$ students, the raw vote totals exceeded this figure, indicating irregular polling submissions. To ensure analytical precision and avoid skewed calculations during class demonstration, the historical data set of $n = 214$ students from the prior semester's Lecture 2 was utilized instead for the complete Goodness-of-Fit computation.