The Chi Square Test: An Introduction

Clinical Research Foundations and the Nature of Categorical Data

  • Medical research continuously collates clinical data, which presents in various formats depending on the nature of the clinical measurement.

  • Quantitative data is the most recognizable form of clinical data, characterized by continuous numerical measurements:

    • For example, straight leg raising (SLR) measurements in subjects capable of raising their legs greater than 0∘0^{\circ} allow researchers to determine average SLR values across two groups.

    • Such continuous quantitative measurements permit comparative hypothesis testing using standard parametric methods, specifically the Student's tt-test.

  • Non-quantitative data cannot be evaluated using standard parametric tests:

    • In clinical assessments, interest frequently centers on subjective outcomes, such as a patient reporting improvement as a binary categorical response ("Yes"\text{"Yes"} or "No"\text{"No"}) following one of two distinct treatment interventions.

    • For binary categorical outcomes, calculating an arithmetic mean improvement for each group is mathematically impossible.

    • Performing a tt-test on categorical proportions is invalid.

    • It is not possible to directly calculate differences between levels of improvement or compute ratios of improvement using continuous scale mechanics.

  • Alternative statistical methodologies are necessary to evaluate categorical data, with the Chi-Square (χ2\chi^2) test of association serving as the primary analytical tool.

  • A working knowledge of categorical association tests is essential for chiropractors and osteopaths to critically appraise clinical and biomedical literature.

Mathematical Framework of the 2x2 Contingency Table

  • The simplest structural framework for evaluating categorical data involves two variables across two categories each, arrayed in a 2×22 \times 2 contingency (frequency) table:

    • Category I constitutes the row variable, divided into type 11 and type 22.

    • Category II constitutes the column variable, divided into type 11 and type 22.

  • The fundamental cell frequencies and marginal totals of a generic 2×22 \times 2 contingency table are defined as follows:

    • aa: The number of individuals who are of type 11 in Category I and type 11 in Category II.

    • bb: The number of individuals who are of type 11 in Category I and type 22 in Category II.

    • cc: The number of individuals who are of type 22 in Category I and type 11 in Category II.

    • dd: The number of individuals who are of type 22 in Category I and type 22 in Category II.

    • n1n_1: The total number of individuals classified as type 11 in Category I, where n1=a+bn_1 = a + b.

    • n2n_2: The total number of individuals classified as type 22 in Category I, where n2=c+dn_2 = c + d.

    • nn: The total sample size across all categories, where n=n1+n2n = n_1 + n_2.

  • Transition from arithmetic means to categorical proportions:

    • p1p_1: The proportion of individuals who are type 11 in Category I and type 11 in Category II.

    • p2p_2: The proportion of individuals who are type 11 in Category I and type 22 in Category II.

    • q1q_1: The proportion of individuals who are type 22 in Category I and type 11 in Category II.

    • q2q_2: The proportion of individuals who are type 22 in Category I and type 22 in Category II.

  • Proportional constraints and distribution across categories:

    • p1+p2=1p_1 + p_2 = 1

    • q1+q2=1q_1 + q_2 = 1

    • The values p1p_1 and p2p_2 represent the proportional distribution of individuals of type 11 in Category I across the levels of Category II.

    • The values q1q_1 and q2q_2 represent the proportional distribution of individuals of type 22 in Category I across the levels of Category II.

  • Null hypothesis formulation:

    • In statistical hypothesis testing, the primary hypothesis evaluated is the null hypothesis (H0H_0), which asserts that nothing happens or that no association exists.

    • In the context of the 2×22 \times 2 table, the null hypothesis posits that p1=q1p_1 = q_1 and p2=q2p_2 = q_2.

    • This formulation specifies that the distribution of individuals across Category II is identical across all levels of Category I, meaning Category II is statistically independent of Category I.

Binary Clinical Trial Analysis: SMT versus IMT for Sciatica

  • Trial structure and clinical cohort parameters:

    • Study cohort: Patients suffering from sciatica allocated to two active management interventions.

    • Intervention 1: Spinal manipulative therapy (SMT) administered over a duration of 66 weeks.

    • Intervention 2: Intermittent motorized traction (IMT) administered over a duration of 66 weeks.

    • Investigative question: Determine whether a statistically significant association exists between subjective clinical improvement and the specific modality of treatment received.

    • Sample allocation: A total of n=390n = 390 sciatica patients participate in the trial, with 190190 patients allocated to IMT and 200200 patients allocated to SMT.

  • Observed patient improvement counts:

    • After 66 weeks, patients report binary subjective improvement ("Yes"\text{"Yes"} versus "No"\text{"No"}).

    • The text initial narrative mentions that for IMT, 8585 reply "Yes"\text{"Yes"} and 8585 reply "No"\text{"No"}, while for SMT, 4545 reply "Yes"\text{"Yes"} and 155155 reply "No"\text{"No"}.

    • The formal contingency table tabulation (Table 2) records the observed frequencies as:

      • IMT / Improved (Yes\text{Yes}): cell a=95a = 95

      • IMT / Not Improved (No\text{No}): cell b=95b = 95

      • IMT Row Total: n1=95+95=190n_1 = 95 + 95 = 190

      • SMT / Improved (Yes\text{Yes}): cell c=45c = 45

      • SMT / Not Improved (No\text{No}): cell d=155d = 155

      • SMT Row Total: n2=45+155=200n_2 = 45 + 155 = 200

      • Total Improved (Yes\text{Yes}): 95+45=14095 + 45 = 140

      • Total Not Improved (No\text{No}): 95+155=25095 + 155 = 250

      • Grand Total: n=190+200=140+250=390n = 190 + 200 = 140 + 250 = 390

  • Calculation of expected frequencies under the null hypothesis:

    • Testing the null hypothesis requires comparing observed values against the values expected if the null hypothesis of no association were true.

    • Overall improvement proportion: Across the entire trial cohort, 140140 out of 390390 patients improved, representing a baseline rate of approximately 36%36\%

    • If no association exists between treatment type and clinical outcome, exactly 36%36\% of patients in each treatment cohort are expected to improve regardless of the therapy received.

    • Expected improvement in IMT cohort: 36%36\% of 190=68190 = 68

    • Expected improvement in SMT cohort: 36%36\% of 200=72200 = 72

    • Overall non-improvement proportion: Across the entire trial cohort, 250250 out of 390390 patients did not improve, representing a baseline rate of approximately 64%64\%

    • If no association exists, 64%64\% of patients in both treatment groups are expected not to improve.

    • Expected non-improvement in IMT cohort: 64%64\% of 190=122190 = 122

    • Expected non-improvement in SMT cohort: 64%64\% of 200=128200 = 128

  • General algebraic formula for expected cell values:

    • Expected value=(Row total)×(Column total)Grand total\text{Expected value} = \frac{(\text{Row total}) \times (\text{Column total})}{\text{Grand total}}

    • Demonstration for the IMT / Improved cell:

    • Expected value=190×140390=68.2≈68\text{Expected value} = \frac{190 \times 140}{390} = 68.2 \approx 68

  • Table 4 Observed and Expected Contingency Table (Expected values in parentheses):

    • IMT / Improved (Yes\text{Yes}): 95 (68)95\,(68)

    • IMT / Not Improved (No\text{No}): 95 (122)95\,(122)

    • IMT Row Total: 190190

    • SMT / Improved (Yes\text{Yes}): 45 (72)45\,(72)

    • SMT / Not Improved (No\text{No}): 155 (128)155\,(128)

    • SMT Row Total: 200200

    • Column Total Improved (Yes\text{Yes}): 140140

    • Column Total Not Improved (No\text{No}): 250250

    • Grand Total: 390390

  • Structural properties of expected frequencies:

    • The sum of expected cell frequencies across any row equals the corresponding observed row total.

    • The sum of expected cell frequencies down any column equals the corresponding observed column total.

    • Expected frequencies are derived strictly under the null hypothesis assumption that treatment type and clinical improvement are independent.

Computational Mechanics and Statistical Inference in 2x2 Tables

  • Mathematical formula for the Chi-Square (χ2\chi^2) test statistic:

    • χ2=∑(Observed−Expected)2Expected\chi^2 = \sum \frac{(\text{Observed} - \text{Expected})^2}{\text{Expected}}

    • Procedure: For every individual cell in the contingency table, subtract the expected value from the corresponding observed value, square the resulting difference, divide this squared value by the expected value, and sum these quotients across all cells.

  • Step-by-step computation for the sciatica clinical trial (Table 5):

    • Cell 1 (IMT / Improved):

      • Observed (OO) = 9595

      • Expected (EE) = 6868

      • O−E=95−68=27O - E = 95 - 68 = 27

      • (O−E)2=272=729(O - E)^2 = 27^2 = 729

      • (O−E)2E=72968=10.72\frac{(O - E)^2}{E} = \frac{729}{68} = 10.72

    • Cell 2 (IMT / Not Improved):

      • Observed (OO) = 9595

      • Expected (EE) = 122122

      • O−E=95−122=−27O - E = 95 - 122 = -27

      • (O−E)2=(−27)2=729(O - E)^2 = (-27)^2 = 729

      • (O−E)2E=729122=5.98\frac{(O - E)^2}{E} = \frac{729}{122} = 5.98

    • Cell 3 (SMT / Improved):

      • Observed (OO) = 4545

      • Expected (EE) = 7272

      • O−E=45−72=−27O - E = 45 - 72 = -27

      • (O−E)2=(−27)2=729(O - E)^2 = (-27)^2 = 729

      • (O−E)2E=72972=10.13\frac{(O - E)^2}{E} = \frac{729}{72} = 10.13

    • Cell 4 (SMT / Not Improved):

      • Observed (OO) = 155155

      • Expected (EE) = 128128

      • O−E=155−128=27O - E = 155 - 128 = 27

      • (O−E)2=272=729(O - E)^2 = 27^2 = 729

      • (O−E)2E=729128=5.70\frac{(O - E)^2}{E} = \frac{729}{128} = 5.70

    • Summation of individual cell components:

      • χ2=10.72+5.98+10.13+5.70=32.53\chi^2 = 10.72 + 5.98 + 10.13 + 5.70 = 32.53

  • Behavioral characteristics of the χ2\chi^2 statistic:

    • When the null hypothesis is true, observed frequencies align closely with expected values, causing the difference terms (O−E)(O - E) to remain small, yielding a small overall χ2\chi^2 value.

    • When the null hypothesis is false, observed values diverge substantially from expected values, resulting in large difference terms and a correspondingly large χ2\chi^2 statistic.

  • Degrees of Freedom (dfdf) formulation:

    • The calculated χ2\chi^2 statistic belongs to a Chi-Square distribution defined by a single parameter known as degrees of freedom.

    • Degrees of freedom depend entirely on the dimensional size of the table being analyzed:

    • Degrees of freedom=(# Rows−1)×(# Columns−1)\text{Degrees of freedom} = (\#\,\text{Rows} - 1) \times (\#\,\text{Columns} - 1)

    • For any standard 2×22 \times 2 contingency table:

    • Degrees of freedom=(2−1)×(2−1)=1×1=1\text{Degrees of freedom} = (2 - 1) \times (2 - 1) = 1 \times 1 = 1

  • The Chi-Square distribution with 11 degree of freedom:

    • Ranges along the horizontal axis continuously from 0→∞0 \rightarrow \infty.

    • The associated pp-value corresponds directly to the total area under the probability density curve lying to the right of the calculated χ2\chi^2 test statistic.

  • Inference and critical threshold evaluation:

    • In a Chi-Square distribution with 11 degree of freedom, the area under the curve to the right of 6.646.64 is less than 0.010.01 (representing a probability of less than 1%1\%

    • Because the calculated test statistic χ2=32.53\chi^2 = 32.53 substantially exceeds 6.646.64, the associated pp-value is confirmed to be p<0.01p < 0.01.

    • Statistical conclusion: The null hypothesis is rejected.

    • Clinical interpretation: The proportion of individuals who received IMT and improved is significantly different from the proportion of individuals who received SMT and improved.

Multi-Category Contingency Tables: The 2x5 Clinical Trial

  • Clinical trial design with multi-level ordinal responses:

    • Many clinical trials employ scales that incorporate more than two levels of improvement.

    • Comparative clinical trial: Evaluation of treatments for acute low back pain.

    • Treatment 1 (Trt 1\text{Trt 1}): Spinal manipulation combined with the application of hot packs.

    • Treatment 2 (Trt 2\text{Trt 2}): Spinal manipulation combined with the application of cold packs.

    • Outcome measurement: A 55-point categorical improvement scale comprising five discrete levels: None, Mild, Noticeable, Definite, and Complete.

  • Observed frequency matrix and marginal totals (Table 6):

    • Trt 1\text{Trt 1}: None = 3939, Mild = 4343, Noticeable = 8989, Definite = 126126, Complete = 8787; Row Total = 384384

    • Trt 2\text{Trt 2}: None = 1212, Mild = 3232, Noticeable = 6565, Definite = 9898, Complete = 6565; Row Total = 272272

    • Column Total for None: 39+12=5139 + 12 = 51

    • Column Total for Mild: 43+32=7543 + 32 = 75

    • Column Total for Noticeable: 89+65=15489 + 65 = 154

    • Column Total for Definite: 126+98=224126 + 98 = 224

    • Column Total for Complete: 87+65=15287 + 65 = 152

    • Grand Total of patients studied: n=384+272=51+75+154+224+152=656n = 384 + 272 = 51 + 75 + 154 + 224 + 152 = 656

  • Null hypothesis for the multi-category scenario:

    • H0H_0: The proportional distribution of clinical improvement across all five outcome levels is identical for both treatments.

  • Expected cell value computations:

    • Expected value for Trt 1 / None\text{Trt 1 / None}:

    • 384×51656=29.85\frac{384 \times 51}{656} = 29.85

    • Expected value for Trt 1 / Mild\text{Trt 1 / Mild}:

    • 384×75656=43.90\frac{384 \times 75}{656} = 43.90

    • The remaining expected values are calculated systematically using the same formula: (Row total)×(Column total)Grand total\frac{(\text{Row total}) \times (\text{Column total})}{\text{Grand total}}.

  • Comprehensive calculation of the multi-category test statistic (Table 7):

    • Trt 1 / None\text{Trt 1 / None}: Observed = 3939, Expected = 29.8529.85, O−E=9.15O - E = 9.15, (O−E)2=83.72(O - E)^2 = 83.72, (O−E)2E=2.80\frac{(O - E)^2}{E} = 2.80

    • Trt 1 / Mild\text{Trt 1 / Mild}: Observed = 4343, Expected = 43.9043.90, O−E=−0.90O - E = -0.90, (O−E)2=0.81(O - E)^2 = 0.81, (O−E)2E=0.02\frac{(O - E)^2}{E} = 0.02

    • Trt 1 / Noticeable\text{Trt 1 / Noticeable}: Observed = 8989, Expected = 90.1590.15, O−E=−1.15O - E = -1.15, (O−E)2=1.32(O - E)^2 = 1.32, (O−E)2E=0.02\frac{(O - E)^2}{E} = 0.02

    • Trt 1 / Definite\text{Trt 1 / Definite}: Observed = 126126, Expected = 131.12131.12, O−E=−5.12O - E = -5.12, (O−E)2=26.21(O - E)^2 = 26.21, (O−E)2E=0.20\frac{(O - E)^2}{E} = 0.20

    • Trt 1 / Complete\text{Trt 1 / Complete}: Observed = 8787, Expected = 88.9888.98, O−E=−1.98O - E = -1.98, (O−E)2=3.92(O - E)^2 = 3.92, (O−E)2E=0.04\frac{(O - E)^2}{E} = 0.04

    • Trt 2 / None\text{Trt 2 / None}: Observed = 1212, Expected = 21.1521.15, O−E=−9.15O - E = -9.15, (O−E)2=83.72(O - E)^2 = 83.72, (O−E)2E=3.96\frac{(O - E)^2}{E} = 3.96

    • Trt 2 / Mild\text{Trt 2 / Mild}: Observed = 3232, Expected = 31.1031.10, O−E=0.90O - E = 0.90, (O−E)2=0.81(O - E)^2 = 0.81, (O−E)2E=0.03\frac{(O - E)^2}{E} = 0.03

    • Trt 2 / Noticeable\text{Trt 2 / Noticeable}: Observed = 6565, Expected = 63.8563.85, O−E=1.15O - E = 1.15, (O−E)2=1.32(O - E)^2 = 1.32, (O−E)2E=0.02\frac{(O - E)^2}{E} = 0.02

    • Trt 2 / Definite\text{Trt 2 / Definite}: Observed = 9898, Expected = 92.8892.88, O−E=5.12O - E = 5.12, (O−E)2=26.21(O - E)^2 = 26.21, (O−E)2E=0.28\frac{(O - E)^2}{E} = 0.28

    • Trt 2 / Complete\text{Trt 2 / Complete}: Observed = 6565, Expected = 63.0263.02, O−E=1.98O - E = 1.98, (O−E)2=3.92(O - E)^2 = 3.92, (O−E)2E=0.06\frac{(O - E)^2}{E} = 0.06

    • Sum of all cell contributions: χ2=7.43\chi^2 = 7.43

  • Degrees of Freedom determination for the 2×52 \times 5 table:

    • Degrees of freedom=(2−1)×(5−1)=1×4=4\text{Degrees of freedom} = (2 - 1) \times (5 - 1) = 1 \times 4 = 4

  • Statistical decision and pp-value evaluation:

    • The pp-value is the area beneath the Chi-Square curve with 44 degrees of freedom lying to the right of 7.437.43.

    • The calculated pp-value is p=0.1148p = 0.1148.

    • Utilizing a standard significance threshold of α=0.05\alpha = 0.05, the pp-value exceeds the threshold (0.1148>0.050.1148 > 0.05).

    • Statistical decision: Fail to reject the null hypothesis.

    • Conclusion: There is no statistically significant difference between the outcomes of the two treatments.

  • Proportional transformation and evaluation of random error (Table 8):

    • Conversion of raw cell frequencies into within-treatment row percentages:

      • Trt 1\text{Trt 1}: None = 10.2%10.2\%, Mild = 11.2%11.2\%, Noticeable = 23.2%23.2\%, Definite = 32.8%32.8\%, Complete = 22.7%22.7\%

      • Trt 2\text{Trt 2}: None = 4.4%4.4\%, Mild = 11.8%11.8\%, Noticeable = 23.9%23.9\%, Definite = 36.0%36.0\%, Complete = 23.9%23.9\%

    • Nominal comparison shows minor apparent differences between individual percentage categories (10.2%≠4.4%10.2\% \neq 4.4\%, 11.2%≠11.8%11.2\% \neq 11.8\%, 23.2%≠23.9%23.2\% \neq 23.9\%, 32.8%≠36.0%32.8\% \neq 36.0\%, 22.7%≠23.9%22.7\% \neq 23.9\%).

  • When the data is evaluated with respect to random error and sampling variability, the observed numerical differences do not provide sufficient statistical evidence of a true underlying difference between spinal manipulation with hot packs versus spinal manipulation with cold packs.

Core Methodological Assumptions and Constraints

  • The Chi-Square test of association relies upon three mandatory structural and probabilistic assumptions that must be strictly satisfied:

    • Assumption 1: Exclusivity of classification. Each observed individual subject must appear in the contingency table once and only once (categories must be mutually exclusive and exhaustive).

    • Assumption 2: Independent observations. The experimental or clinical outcome for each individual subject must be completely independent of all other individual subjects in the sample.

    • Assumption 3: Minimum expected cell count threshold. In the generated table of expected values, at least 80%80\% of all expected cell values must be strictly greater than 55.

Provenance and Bibliographic Citations

  • Authorship and institutional affiliations:

    • Antony Ugoni, B.Sc. (Hons), Department of Public Health & Community Medicine, The University of Melbourne, Parkville, Victoria, 3052, Australia.

    • Bruce F. Walker, D.C., M.P.H., Private Practice, 33 Wantirna Road, Ringwood, Victoria, 3134, Australia (Telephone: 03 879 5555).

  • Publication data:

    • Published in COMSIG Review, Volume 4, Number 3, November 1995, pages 61–64.

    • Key indexing terms: Chi square, chiropractic, osteopathy.

  • Formal literature citations:

    • Ugoni A. On the subject of hypothesis testing. COMSIG Review, 1993; 2(2): 45–48.

    • Neave HR. Statistics Tables for Mathematicians, Engineers, Economists, and the Behavioural and Management Sciences. Unwin Hyman Ltd, 1988: 42–43.