Hypothesis Testing: Chi-Square and Two-Sample t-Test Calculations

Foundations of Hypothesis Testing and Statistical Inference

• Core Objective of Inference:

  • In empirical research, the fundamental goal is to determine whether an independent variable ($X$) and a dependent variable ($Y$) covary systematically within an entire population.
  • Because population parameters are inherently unknown and directly unobservable, empirical research relies on random sampling to draw a representative subset and induce population parameters from sample statistics.

• Formulating Testable Hypotheses:

  • Theoretical propositions must be operationalized and narrowed down into concrete, quantifiable, and testable empirical hypotheses.
  • Hypothesis testing may involve:
  • Comparing differences between two distinct sample groups.
  • Comparing a single observed sample against a hypothetical benchmark or target value.
  • Every research hypothesis requires the formulation of a corresponding null hypothesis ($H_0$), which posited the exact opposite condition (i.e., that no relationship, covariation, or difference exists in the underlying population).

• The Four-Step Logic of Hypothesis Testing:

  • Step 1: Determine the theoretical expectation under the condition that the null hypothesis ($H_0$) is true.
  • Step 2: Compare the observed empirical sample data directly to what would be expected if the null hypothesis were true.
  • Step 3: Assess the magnitude of the discrepancy; a greater difference between observation and expectation provides greater confidence that the observed sample relationship is not driven by chance alone.
  • Step 4: Reject the null hypothesis if the probability of observing such a relationship strictly due to chance is less than $5\%$ ($p < 0.05$).

The Nature and Interpretation of the p-Value

• Definition and Functional Meaning:

  • The $p$-value quantifies the probability of observing an empirical sample relationship as strong as, or stronger than, the observed data purely as a result of stochastic variation, sampling error, or random chance alone.
  • The standard decision threshold across introductory research methodology is a significance level of $5\%$ ($\alpha = 0.05$).
  • If the $p$-value is less than $5\%$ ($p < 0.05$), the finding is declared statistically significant, leading to the formal rejection of the null hypothesis.
  • If the $p$-value is greater than or equal to $5\%$ ($p \ge 0.05$), the finding is not statistically significant, meaning the null hypothesis cannot be rejected.

• Crucial Misconceptions Regarding the p-Value:

  • Measurement Quality: The $p$-value provides zero information regarding whether the measurement instruments, operational definitions, or data collection processes are of high quality or low quality.
  • Causality: The $p$-value does not establish whether an observed relationship is causal; a statistically significant relationship merely indicates that covariation is unlikely to be a random artifact.
  • Substantive Significance: A $p$-value does not convey the substantive magnitude or practical importance of an effect; it exclusively functions as a decision instrument to discern random fluctuations from systematic sample-to-population inferences.

Tabular Analysis and the Chi-Square Test of Independence

• Analytical Context and Variable Types:

  • Tabular analysis using the Chi-square ($\chi^2$) test is conducted when both the independent variable ($X$) and the dependent variable ($Y$) are categorical.
  • The Chi-square test evaluates whether the distribution of observed cell frequencies deviates significantly from the expected frequencies predicted under the assumption of statistical independence (the null hypothesis).
  • Chi-square tests of independence are inherently one-tailed tests, evaluating whether the accumulated squared deviations exceed critical thresholds in the right tail of the $\chi^2$ distribution.

• Mathematical Formula:

  • The Chi-square statistic is calculated by summing the normalized squared differences across all table cells:   $\chi^2 = \sum \frac{(O - E)^2}{E}$
  • Where $O$ represents the observed frequency in a specific cell, and $E$ represents the expected frequency for that cell under the null hypothesis.

• Degrees of Freedom ($df$):

  • For a contingency table, the degrees of freedom are calculated as:   $df = (r - 1) \times (c - 1)$
  • Where $r$ denotes the total number of rows and $c$ denotes the total number of columns.

• Comprehensive Worked Example: Sleep Medication:

  • Variables and Data Setup:
  • Independent variable ($X$): Medication status (Took medication vs. Did not take medication).
  • Dependent variable ($Y$): Sleep quality (Slept better vs. Did not sleep better).
  • Overall distribution of outcomes: $30\%$ did not sleep well; $70\%$ slept well.
  • For the "Did not sleep better" outcome ($N = 6$ total):
    • Observed: Did not take medication = $5$
    • Observed: Took medication = $1$
  • For the "Slept better" outcome ($N = 14$ total):
    • Observed: Did not take medication = $5$
    • Observed: Took medication = $9$
  • Derivation of Expected Frequencies ($E$):
  • Under the null hypothesis ($H_0$), medication status exerts no influence on sleep outcome; responses should divide evenly across the treatment categories.
  • For "Did not sleep better" ($6$ total respondents), expected cell values are split across the $2$ groups:     $E = \frac{6}{2} = 3$
  • For "Slept better" ($14$ total respondents), expected cell values are split across the $2$ groups:     $E = \frac{14}{2} = 7$
  • Step-by-Step Chi-Square Calculation:
  • Applying the formula cell-by-cell:     $\chi^2 = \frac{(5 - 3)^2}{3} + \frac{(1 - 3)^2}{3} + \frac{(5 - 7)^2}{7} + \frac{(9 - 7)^2}{7}$$\chi^2 = \frac{2^2}{3} + \frac{(-2)^2}{3} + \frac{(-2)^2}{7} + \frac{2^2}{7}$$\chi^2 = \frac{4}{3} + \frac{4}{3} + \frac{4}{7} + \frac{4}{7}$$\chi^2 = \frac{8}{3} + \frac{8}{7}$$\chi^2 = \frac{56 + 24}{21} = \frac{80}{21} \approx 3.81$
  • Degrees of Freedom Calculation:$df = (2 - 1) \times (2 - 1) = 1 \times 1 = 1$
  • Statistical Inference and Decision:
  • For $df = 1$ at the $5\%$ significance level ($\alpha = 0.05$), the theoretical critical value is $3.84$ (or $3.841$).
  • Comparing the calculated statistic to the critical threshold: $3.81 < 3.84$.
  • Because the test statistic does not exceed the critical value, the observed result falls to the left of the critical rejection region.
  • Consequently, the probability of observing these data due to chance alone exceeds $5\%$.
  • The null hypothesis cannot be rejected; the relationship between medication and sleep quality is not statistically significant at the $5\%$ level.

• Additional Chi-Square Critical Values for Reference:

  • For $df = 2$ at $\alpha = 0.05$: Critical value is $5.99$.
  • For $df = 25$ at $\alpha = 0.05$: Value is determined by referencing row $25$ on the Chi-square table.
  • Supplementary example with calculated $\chi^2 = 3.27$ ($df = 1$):
  • Calculated value $3.27 < 3.841$.
  • The tail probability represents approximately an $8\%$ or $9\%$ chance of occurrence under random variation.
  • Because $8\% \text{ to } 9\% > 5\%$, the null hypothesis cannot be rejected.

Difference of Means Testing and the t-Statistic

• Analytical Context and Variable Types:

  • Difference of means testing is performed when the independent variable ($X$) is categorical (typically two groups) and the dependent variable ($Y$) is continuous.
  • The procedure compares the sample means of two groups to evaluate whether the difference between them reflects a genuine difference in population means.

• General Formula for the Two-Sample t-Statistic:

  • The $t$-statistic divides the difference between sample means by the standard error of that difference:   $t = \frac{\bar{X}_1 - \bar{X}_2}{SE_{(\bar{X}_1 - \bar{X}_2)}}$

• Degrees of Freedom for Difference of Means:

  • The total degrees of freedom correspond to the sum of the degrees of freedom for each independent sample:   $df = (n_1 - 1) + (n_2 - 1)$
  • Simplified as:   $df = n_1 + n_2 - 2$

• Comprehensive Worked Example: Difference of Means Calculation:

  • Sample Parameters:
  • Sample 1: Size $n_1 = 24$, Sample Mean $\bar{X}_1 = 30$, Sample Standard Deviation $s_1 = 60$
  • Sample 2: Size $n_2 = 48$, Sample Mean $\bar{X}_2 = 48$, Sample Standard Deviation $s_2 = 40$
  • Degrees of Freedom:$df = 24 + 48 - 2 = 70$
  • Standard Error Calculation:
  • The standard error in the denominator evaluates to approximately $11.877$.
  • Structured computational breakdown: the expression decomposes into two components where the second component simplifies via:     $\sqrt{\frac{3}{48}} = \sqrt{\frac{1}{16}} = \frac{1}{4}$
  • Multiplying the leading factor ($47.51$) by $\frac{1}{4}$ (or dividing by $4$) produces the standard error:     $\frac{47.51}{4} \approx 11.877$
  • t-Statistic Calculation:
  • Difference in sample means:     $\bar{X}_1 - \bar{X}_2 = 30 - 48 = -18$
  • Computing $t$:     $t = \frac{-18}{11.877} \approx -1.516$

Directional Hypotheses and the Critical Value Decision Framework

• One-Tailed vs. Two-Tailed Hypotheses:

  • Two-Tailed Test (Non-Directional):
  • Used when the research hypothesis predicts that two groups differ, but does not specify the direction of the difference ($H_1: \mu_1 \ne \mu_2$).
  • The null hypothesis specifies equality ($H_0: \mu_1 = \mu_2$).
  • The total $5\%$ alpha level is divided equally between the two tails ($2.5\%$ in the lower tail and $2.5\%$ in the upper tail).
  • Critical values must be located under the $2.5\%$ ($0.025$) column on the $t$-table.
  • One-Tailed Test (Directional):
  • Used when the research hypothesis explicitly predicts the direction of the effect ($H_1: \mu_1 > \mu_2$ or $H_1: \mu_1 < \mu_2$).
  • The null hypothesis encompasses the opposite condition and equality ($H_0: \mu_1 \le \mu_2$ or $H_0: \mu_1 \ge \mu_2$).
  • The entire $5\%$ alpha level is placed into a single tail.
  • Critical values must be located under the $5\%$ ($0.05$) column on the $t$-table.

• Handling Negative t-Statistics and Distribution Symmetry:

  • The Student's $t$-distribution is perfectly symmetric around zero.
  • Standard $t$-distribution tables display critical values as positive numbers.
  • When evaluating an empirically calculated negative $t$-statistic (such as $-1.516$), two mathematically equivalent approaches apply:
  • Absolute Value Approach: Ignore the negative sign of the calculated $t$-statistic and compare its absolute magnitude directly against the positive critical value ($|t_{\text{calc}}| > t_{\text{crit}}$).
  • Negative Benchmark Approach: Assign a negative sign to the critical value from the table and assess whether the calculated statistic is more extreme in the negative direction ($t_{\text{calc}} < -t_{\text{crit}}$).

• Comparison and Final Decision for the Worked Example:

  • Evaluated against a one-tailed $5\%$ threshold with $df = 70$ (using reference benchmark $df = 60$, where critical value $t_{\text{crit}} = 1.67$):
  • The absolute value of the calculated statistic is $|-1.516| = 1.516$.
  • Comparison: $1.516 < 1.67$ (or $-1.516 > -1.67$).
  • The calculated $t$-statistic does not fall within the extreme rejection region.
  • The null hypothesis cannot be rejected; the difference in means is not statistically significant at the $5\%$ level.
  • Large Degrees of Freedom Benchmarks:
  • When degrees of freedom exceed table listings (e.g., $df = 110$ located between $100$ and higher values), critical values shift negligibly (e.g., $1.66$ to $1.65$). These marginal variations do not affect inference unless the statistic falls on the exact boundary.

Scope of Coverage and Examination Specifications

• Included and Excluded Curriculum Content:

  • Included Topics (Chapters 1 through 8):
  • Tabular analysis and Chi-square ($\chi^2$) testing.
  • Difference of means and $t$-statistic calculations.
  • Correlation coefficients.
  • Excluded Topics:
  • Probit models.
  • Logit models.
  • Bivariate regression analysis.

• Examination Structure and Timing:

  • Total number of exam questions: 31 questions.
  • Total number of calculation questions: Approximately 5 questions.
  • Calculation time estimate: Under 2 minutes per calculation problem.
  • Question sources: All exam questions are derived directly from textbook problems, practice quizzes, and lecture discussions; no novel question formats appear on the assessment.

Questions & Discussion

• Question: Does nullifying the null hypothesis require subtracting the calculated test statistic from the critical value or $p$-value?

  • Response: No subtraction is required. A direct pairwise comparison between the calculated test statistic and the critical value is all that is necessary. Alternatively, if subtracting the critical value from the test statistic ($\text{Statistic} - \text{Critical Value}$), a positive difference warrants rejecting the null hypothesis, whereas a negative difference indicates failure to reject. The numerical distance between the statistic and the critical value does not indicate "how significant" a finding is; statistical significance at a fixed $\alpha = 0.05$ level is a binary threshold.

• Question: How should one proceed if a slide contains a typographical error in cell numbers (such as displaying 209)?

  • Response: Use the numbers explicitly provided on the assessment materials to carry out calculations directly, avoiding confusion during calculations.

• Question: In tabular analysis, does one plug the sum of cells into another formula, or immediately identify degrees of freedom?

  • Response: The sum of squared deviations divided by expected values across all cells yields the Chi-square statistic directly. The degrees of freedom are calculated separately using row and column counts ($(r-1)(c-1)$) to find the critical value from the table.

• Question: Why is the final $t$-statistic sometimes written without a negative sign?

  • Response: The calculated value is strictly negative ($-1.516$). Because the $t$-distribution is symmetric, whether the sign is retained or absolute values are evaluated depends on whether a one-tailed or two-tailed comparison is applied; both yield identical conclusions when evaluated in the proper tail direction.

• Question: Will students be instructed whether to perform a one-tailed or two-tailed test on the exam?

  • Response: The problem context will define the appropriate test. If the hypothesis specifies a directional relationship (e.g., predicting that Sample A has a higher mean than Sample B), a one-tailed test must be used. If the hypothesis specifies only an unspecified difference, a two-tailed test must be used.

• Question: Will students be required to calculate critical values that are not explicitly printed on the chart?

  • Response: No. The reference critical value table will be provided. The student only needs to locate the appropriate critical value based on the degrees of freedom and whether the test is one-tailed or two-tailed at the $5\%$ level.

• Question: Are expected cell values ($E$) always provided in Chi-square questions, or must they be calculated?

  • Response: Both question formats appear: in some instances, expected values are provided directly; in other instances, students must compute expected values based on row, column, or category totals.