Week 1, Day 3 Part 2-Contingency Tables and Probability Analysis

Overview and Definition of Contingency Tables

• Definition: A contingency table is a statistical tool used to display the number of cases (frequencies) for combinations of two categorical variables.

• Primary Function: These tables help in identifying relationships or associations between different categories.

• Example Categories:

  • Biological Sex (Male vs. Female)

  • Animal Preference (Cats vs. Dogs)

Structure of a $2 \times 2$ Contingency Table

• A $2 \times 2$ table is the most basic form of a contingency table, tracking two outcomes across two groups.

• Cell Layout:

  • Group 1: Contains values $a$ (Outcome A) and $b$ (Outcome B).

  • Group 2: Contains values $c$ (Outcome A) and $d$ (Outcome B).

• Margins:

  • Row Totals: The sum of the frequencies across a row. For Group 1, the total is $a + b$. For Group 2, the total is $c + d$.

  • Column Totals: The sum of the frequencies down a column. For Outcome A, the total is $a + c$. For Outcome B, the total is $b + d$.

  • Grand Total ($n$): The sum of all cases in the table, calculated as $a + b + c + d = n$.

• Key Concept: The margins specifically show the totals for each individual row or column variable.

Case Study: Study Habits vs. Pass Rate

• This example involves a class of $100$ students, categorized by study duration (greater than/equal to $6\,\text{hrs}$ vs. less than $6\,\text{hrs}$) and exam result (Pass vs. Not Pass).

• Table Data:

  • Studies $\ge 6\,\text{hrs}$: $36$ Pass, $9$ Not Pass, Total = $45$.

  • Studies < 6\,\text{hrs}: $24$ Pass, $31$ Not Pass, Total = $55$.

  • Column Totals: Total Pass = $60$, Total Not Pass = $40$.

  • Grand Total: $100\,\text{students}$.

Marginal, Joint, and Conditional Probabilities

• Marginal Probabilities: These summarize each variable separately by looking at the totals in the margins relative to the grand total.

  • Proportion of students who passed: $\frac{60}{100} = 0.60$

  • Proportion of students who study $\ge 6\,\text{hrs}$: $\frac{45}{100} = 0.45$

• Joint Probabilities: These represent the intersection of two events, or the likelihood of two conditions being met simultaneously.

  • Probability of a student studying $\ge 6\,\text{hrs}$ AND passing: $\frac{36}{100} = 0.36$

  • Probability of a student studying < 6\,\text{hrs} AND not passing: $\frac{31}{100} = 0.31$

• Conditional Probabilities: This is defined as the probability of one subgroup (event) occurring, given the knowledge of a specific group (condition).

  • Pass Rate given $\ge 6\,\text{hrs}$: $\frac{36}{45} = 80\%$

  • Pass Rate given < 6\,\text{hrs}: $\frac{24}{55} \approx 43.6\%$

Simpson’s Paradox and Data Aggregation

• Definition: Simpson's Paradox occurs when combining data from different groups results in a misleading or counterintuitive conclusion compared to analyzed individual groups.

• Hiring Process Example:

  • Department A Statistics: $80$ Hired, $20$ Not Hired, $100$ Total Applicants.

  • Department B Statistics: $10$ Hired, $90$ Not Hired, $100$ Total Applicants.

  • Aggregated Data: $90$ Total Hired, $110$ Total Not Hired, $200$ Total Overall Applicants.

• Hiring Rates per Department:

  • $\text{Hiring Rate}_A = \frac{80}{80 + 20} = \frac{80}{100} = 0.80 \, (80\%)$

  • $\text{Hiring Rate}_B = \frac{10}{10 + 90} = \frac{10}{100} = 0.10 \, (10\%)$

• Overall Hiring Rate Analysis:

  • $\text{Total Hired} = 80 + 10 = 90$

  • $\text{Total Applicants} = 100 + 100 = 200$

  • $\text{Overall Hiring Rate} = \frac{90}{200} = 0.45 \, (45\%)$

• Observation: The combined hiring rate ($45\%\text{}$) is numerically between the individual department rates ($80\%\text{}$ and $10\%\text{}$). However, aggregation can mask the extreme differences between the groups.

• Analytical Takeaway: It is critically important to analyze data by considering underlying groups rather than relying solely on aggregated totals.

Key Takeaways

• Contingency tables effectively summarize the relationships between two categorical variables.

• Marginal, joint, and conditional probabilities offer distinct views of the same dataset and should be calculated to understand the data's nuances.

• Analysts must be vigilant regarding confounding results (such as Simpson's Paradox) that can lead to incorrect data interpretations.