Comprehensive Guide to Statistical Inference: Two-Sample Analysis, Confidence Intervals, and Quantitative Summaries

Dolphin Therapy Study on Depression

• Research Question & Context:

  • The study evaluates whether swimming with dolphins reduces depression symptoms compared to a control group (swimming without dolphins).
  • The observed sample difference in improvement rates between the two groups is $46.7\%$.
  • The core statistical question is whether an observed difference of $46.7\%$ is statistically significant or if it could plausibly occur by random chance under the null hypothesis ($\pi_1 - \pi_2 = 0$).

• Hypotheses:

  • Null Hypothesis ($H_0$): Swimming with dolphins has no effect on depression improvement rates ($\pi_{\text{dolphin}} - \pi_{\text{control}} = 0$). Any observed difference in the sample is a fluke resulting from small sample size, meaning improvers would have improved regardless of group assignment.
  • Alternative Hypothesis ($H_a$): Swimming with dolphins increases depression improvement rates ($\pi_{\text{dolphin}} - \pi_{\text{control}} > 0$).

• Sample Data Breakdown:

  • Total subjects: $n = 30$.
  • Total improvers across both groups: $13$.
  • Total non-improvers across both groups: $17$.
  • Dolphin group ($n_1 = 15$): $10$ improvers, $5$ non-improvers ($66.7\%$ improvement rate).
  • Control group ($n_2 = 15$): $3$ improvers, $12$ non-improvers ($20.0\%$ improvement rate).
  • Observed difference: $66.7\% - 20.0\% = 46.7\%$ ($0.467$).

• Card Simulation Methodology (3S Strategy):

  • To test what differences occur naturally under $H_0$, simulation is conducted using physical or computer cards:
  • $13$ cards are labeled "improver" and $17$ cards are labeled "non-improver".
  • Cards are shuffled and randomly dealt into two piles of $15$ (representing the dolphin group and control group).
  • The difference in proportions between the two simulated piles ($\hat{p}_1 - \hat{p}_2$) is computed and stored.
  • Sample Simulation Trials:
  • Trial 1: Dolphin group $= 40.0\%$, Control group $= 46.7\%$, Difference $= -6.7\%$.
  • Trial 2: Difference $= +20.0\%$.
  • Trial 3: Difference $= +6.7\%$.
  • Simulation Results:
  • Across $40$ initial simulations: Most differences fall in the range of $-20\%$ to $+20\%$. Differences up to $40\%$ are rare. Only $1$ out of $30$ simulations ($\approx 3\%$ ) yielded a difference $\ge 46.7\%$.
  • Across $1000$ full simulations:
    • Mean of simulated differences $= -0.005$.
    • Standard error ($SE$) of simulated differences $= 0.185$.
    • Exactly $13$ out of $1000$ simulations produced a difference $\ge 0.467$.
    • Empirical $p\text{-value} = \frac{13}{1000} = 0.013$ ($1.3\%$ ).

• Standardized Test Statistic ($z$):

  • $z = \frac{\text{Observed Difference} - \text{Null Expected Value}}{SE} = \frac{0.467 - 0}{0.185} = 2.52$.
  • Both the $p\text{-value}$ ($0.013 < 0.05$) and the $z\text{-score}$ ($2.52 > 1.96$) provide strong evidence against $H_0$. The observed difference is statistically significant.

• $95\%$ Confidence Interval for Dolphin Study:

  • Formula: $\text{Statistic} \pm \text{Multiplier} \times SE$
  • Multiplier for $95\%$ confidence $= 1.96$
  • Calculation: $0.467 \pm 1.96 \times 0.185 = 0.467 \pm 0.363$
  • Interval: $[0.104, 0.830]$ or $10.4\%$ to $83.0\%$
  • Conclusion: Zero ($0$) is excluded from the interval. We are $95\%$ confident that swimming with dolphins increases the population improvement rate by between $10.4\%$ and $83.0\%$ compared to swimming without dolphins.

• Causation vs. Generalization:

  • Causation: Yes, causal conclusions are justified because subjects were randomly assigned to treatment groups in a randomized controlled experiment. Random assignment equalizes potential confounding variables between groups.
  • Generalization: Generalizing to all depressed individuals is questionable. Subjects were not selected via simple random sampling; outpatients aged $18$ to $65$ with mild-to-moderate depression were recruited through internet, radio, newspaper, and hospital announcements. They may not represent the general population.

Prediction vs. Causation and Confounding Factors

• Core Distinction:
  • When the primary goal is prediction of a response variable $Y$ rather than establishing causation ($X$ causes $Y$), confounding factors do not invalidate the utility of a predictor variable.
• Examples:
  • Smoking and Liver Cancer:
  • Smoking correlates with liver cancer primarily due to confounding (smokers drink more alcohol, and alcohol directly causes liver damage/cancer).
  • If the objective is solely predicting whether an individual will develop liver cancer or estimating lifespan, smoking remains a valuable predictor variable even if non-causal.
  • Temperature and Wildfires:
  • Hotter days exhibit higher wildfire incidence due to direct thermal mechanisms (easier ignition/spread) and confounding human mechanisms (higher temperatures prompt more camping, increasing illegal campfires).
  • For wildfire forecasting models, temperature serves as a highly effective predictive input regardless of which mechanism dominates.

Comparing Two Proportions: Parental Smoking and Infant Gender

• Study Background (Fukuda et al., 2002):

  • Investigated whether parental smoking influences the sex ratio of offspring in a Japanese cohort.
  • Both parents smokers: $n_1 = 565$ births, $255$ male births ($45.1\%$ male).
  • Both parents non-smokers: $n_2 = 3602$ births, $1975$ male births ($54.8\%$ male).
  • Combined cohort: $n = 4170$ total births, $2230$ male births ($53.5\%$ male).

• Hypotheses:

  • Null Hypothesis ($H_0$): $\pi_{\text{smoking}} - \pi_{\text{nonsmoking}} = 0$ (Parental smoking is not associated with child's gender).
  • Alternative Hypothesis ($H_a$): $\pi_{\text{smoking}} - \pi_{\text{nonsmoking}} \neq 0$ (Two-sided test).

• Summary Statistic:

  • $\hat{p}_{1} - \hat{p}_{2} = \hat{p}_{\text{nonsmokers}} - \hat{p}_{\text{smokers}} = 0.548 - 0.451 = 0.097$ ($9.7\%$ ).

• Simulation-Based Analysis ($1000$ Replications):

  • Mean difference under $H_0 = 0.00021$
  • Standard error ($SE$) $= 0.023$ ($2.3\%$ )
  • Empirical $p\text{-value} = \frac{0}{1000} = 0.000$
  • Simulated $z\text{-statistic} = \frac{0.097 - 0}{0.023} = 4.22$

• Theory-Based Standard Error Formulas:

  • For Hypothesis Testing (Pooled Proportion):
  • Assumes $H_0$ is true ($\pi_1 = \pi_2 = \pi$).
  • Pooled proportion ($\hat{p}$): $\hat{p} = \frac{\text{Total Male Births}}{\text{Total Births}} = \frac{255 + 1975}{565 + 3602} = \frac{2230}{4170} = 0.535$
  • Test Standard Error formula: $SE_{\text{test}} = \sqrt{\hat{p}(1 - \hat{p}) \left(\frac{1}{n_1} + \frac{1}{n_2}\right)}$
  • Calculation: $SE_{\text{test}} = \sqrt{0.535 \times (1 - 0.535) \times \left(\frac{1}{3602} + \frac{1}{565}\right)} = 0.02256$
  • Theoretical $z\text{-statistic}$: $z = \frac{0.097 - 0}{0.02256} = 4.30$
  • Theoretical $p\text{-value}$: $2 \times (1 - \text{pnorm}(4.30)) = 0.0000171$ ($0.00171\%$ ).
  • For Confidence Intervals (Unpooled Proportions):
  • Does not assume $H_0$ is true; estimates individual group variances.
  • Confidence Interval Standard Error formula: $SE_{\text{CI}} = \sqrt{\frac{\hat{p}_1(1 - \hat{p}_1)}{n_1} + \frac{\hat{p}_2(1 - \hat{p}_2)}{n_2}}$
  • Calculation: $SE_{\text{CI}} = \sqrt{\frac{0.548 \times (1 - 0.548)}{3602} + \frac{0.451 \times (1 - 0.451)}{565}} = 0.0224$
  • $95\%$ Confidence Interval ($\text{multiplier} = 1.96$): $0.097 \pm 1.96 \times 0.0224 = 0.097 \pm 0.044 \rightarrow [0.053, 0.141]$ ($5.3\%$ to $14.1\%$ ).
  • $99\%$ Confidence Interval ($\text{multiplier} = 2.576$): $0.097 \pm 2.576 \times 0.0224 = 0.097 \pm 0.058 \rightarrow [0.039, 0.155]$ ($3.9\%$ to $15.5\%$ ).

• Validity Conditions for Theoretical Proportions Methods:

  • At least $10$ observations in each of the four cells of the $2 \times 2$ cross-tabulation table (at least $10$ boys and $10$ girls in both smoking and non-smoking groups).

• Methodological Evaluation & Biases:

  • This study was strictly observational; subjects self-selected smoking behavior.
  • James (2004) highlighted confounding factors such as social factors, diet, environmental exposures, or stress.
  • Publication Bias: Hundreds of sex-ratio studies are performed; non-significant findings often remain unpublished, whereas rare anomalous flukes are accepted for publication.
  • Data Anomalies: An overall male proportion of $53.5\%$ ($46.5\%$ female) is unusually high relative to global human baseline birth ratios, suggesting possible recording or sampling errors.

Summary Statistics for Quantitative Data

• Visual Summaries:

  • Dot Plots: Display individual data points on a numeric scale; stacking vertical dots highlights distributional shape.
  • Box Plots: Graphical representation of the five-number summary.

• Five-Number Summary & Definitions:

  1. Minimum: Lowest recorded observation.
  2. Lower Quartile ($Q_1$ / $25\text{th}$ Percentile): Median of the lower half of data (at or below overall median).
  3. Median ($Q_2$ / $50\text{th}$ Percentile): Middle boundary value splitting sorted dataset $50/50$ .
  4. Upper Quartile ($Q_3$ / $75\text{th}$ Percentile): Median of the upper half of data.
  5. Maximum: Highest recorded observation.

• Interquartile Range ($IQR$):

  • Formula: $IQR = Q_3 - Q_1$
  • Measures spread/variability in quantitative data. It is a single scalar value, not an interval.

• Quartile Calculation Conventions:

  • Dataset: $1, 3, 7, 7, 8, 9, 12, 14$ ($n = 8$ )
  • Median ($Q_2$): Boundary falls between $7$ and $8$ $\rightarrow \frac{7 + 8}{2} = 7.5$
  • Lower Quartile ($Q_1$): Median of lower half ($1, 3, 7, 7$) $\rightarrow \frac{3 + 7}{2} = 5.0$
  • Upper Quartile ($Q_3$): Median of upper half ($8, 9, 12, 14$) $\rightarrow \frac{9 + 12}{2} = 10.5$
  • $IQR = 10.5 - 5.0 = 5.5$
  • R Software Convention: Standard course convention uses type = 2 (averaging middle values during ties). type = 1 takes the lower minimum boundary point on ties.

• Case Study: Old Faithful Geyser Inter-Eruption Times:

  • Vertically stacked dot plots and box plots allow direct comparison of eruption intervals between $1978$ and $2003$
  • $1978$ Dataset Summary: $Q_1 = 58\,\text{min}$, $Q_3 = 81\,\text{min}$, $IQR = 81 - 58 = 23\,\text{min}$, Minimum $= 42\,\text{min}$, Maximum $= 95\,\text{min}$.
  • $2003$ Dataset Summary: $Q_1 = 87\,\text{min}$, $Q_3 = 98\,\text{min}$, $IQR = 98 - 87 = 11\,\text{min}$, Median $\approx 92\,\text{min}$.
  • Comparison: In $1978$, eruption intervals were shorter on average but exhibited far greater variability ($IQR = 23$) compared to $2003$ ($IQR = 11$).

• Box Plot Construction & Outlier Rules:

  • Box: Extends from $Q_1$ to $Q_3$; vertical bar marks $Q_2$ (median).
  • Outlier Threshold Boundary: Any observation lower than $Q_1 - 1.5 \times IQR$ or higher than $Q_3 + 1.5 \times IQR$ is defined as an outlier and plotted as an individual asterisk/symbol.
  • Whiskers: Extend outwards to the most extreme data points that are not outliers (do not extend blindly to the exact $1.5 \times IQR$ fence boundary).

Comparing Two Means: Breastfeeding and Infant Intelligence

• Study Background (Pediatrics, 1999):

  • Investigated whether breastfed infants differ in cognitive intelligence at age $4$ compared to bottle-fed infants.
  • Cohort: $323$ children recruited at birth ($1980\text{--}1981$) across $4$ Western Michigan hospitals.
  • Measure: General Cognitive Index ($GCI$) at age $4$ (known to be normally distributed in the general population).
  • Observational study: Parents selected feeding method.

• Sample Data Breakdown:

  • Breastfed group ($n_1 = 237$): Mean $\bar{x}_1 = 105.3$, Sample SD $s_1 = 14.5$
  • Bottle-fed group ($n_2 = 85$): Mean $\bar{x}_2 = 100.9$, Sample SD $s_2 = 14.0$
  • Difference in sample means: $\bar{x}_1 - \bar{x}_2 = 105.3 - 100.9 = 4.4$

• Inference & Calculations:

  • Standard Error formula for two independent means: $SE = \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}$
  • Calculation: $SE = \sqrt{\frac{14.5^2}{237} + \frac{14.0^2}{85}} = \sqrt{\frac{210.25}{237} + \frac{196.0}{85}} = \sqrt{0.8871 + 2.3059} = 1.79$
  • Standardized Test Statistic ($t$): $t = \frac{(\bar{x}_1 - \bar{x}_2) - 0}{SE} = \frac{4.4 - 0}{1.79} = 2.46$
  • $p\text{-value} = 0.014$ ($1.4\%$ ).
  • $95\%$ Confidence Interval ($\text{multiplier} = 1.96$): $4.4 \pm 1.96 \times 1.79 = 4.4 \pm 3.51 \rightarrow [0.89, 7.91]$
  • Conclusion: Statistically significant evidence ($p = 0.014 < 0.05$; CI excludes $0$) that breastfed children have a significantly higher average GCI score at age $4$ than bottle-fed children.

• Confounding Variables in Breastfeeding Study:

  • Maternal education level (more educated mothers may be more likely to breastfeed and provide greater intellectual stimulation).
  • Maternal health status and medication use (unhealthy mothers may be physically unable to breastfeed).
  • Time spent directly interacting with infant during feeding.
  • Infant baseline health/appetite differences.

Comprehensive Decision Matrix for Standard Errors and Confidence Intervals

• Sample Size Thresholds:

  • Quantitative data: Sample size $n \ge 30$ is considered large.
  • Binary data: At least $10$ successes and $10$ failures ($n\hat{p} \ge 10$ and $n(1-\hat{p}) \ge 10$) is considered large.

• Confidence Interval Multipliers (Normal Distribution):

  • $90\%$ Confidence Level: Multiplier $= 1.645$
  • $95\%$ Confidence Level: Multiplier $= 1.96$
  • $99\%$ Confidence Level: Multiplier $= 2.576$

• One-Sample Quantitative Data:

  • Formula: $\bar{x} \pm z^* \times \frac{s}{\sqrt{n}}$
  • If $n < 30$ and population is normal with unknown $\sigma$, substitute $t^*$ (from t-distribution) for $z^*$ .

• One-Sample Binary Data:

  • Formula: $\hat{p} \pm 1.96 \times \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$
  • Note: Binary proportions are mathematically equivalent to numeric mean formulas where data values are strictly $0$ and $1$, with $\bar{x} = \hat{p}$ and $s = \sqrt{\hat{p}(1-\hat{p})}$.

• Two-Sample Quantitative Data:

  • Formula: $(\bar{x}_1 - \bar{x}_2) \pm 1.96 \times \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}$

• Two-Sample Binary Data:

  • Confidence Interval Formula: $(\hat{p}_1 - \hat{p}_2) \pm 1.96 \times \sqrt{\frac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \frac{\hat{p}_2(1-\hat{p}_2)}{n_2}}$
  • Hypothesis Testing Formula: $z = \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\hat{p}(1-\hat{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}$ where $\hat{p} = \frac{x_1 + x_2}{n_1 + n_2}$.

Review Problems and Exam Questions

• Problem 1: NCIS Viewing and Mortality:

  • Scenario: NCIS was a top-rated TV show in $2022$. A study finds that people who watch NCIS are much more likely to die than non-viewers. Can we conclude NCIS is dangerous to watch?
  • Answer: No. Age is a severe confounding factor in this observational study; the median age of an NCIS viewer is $61\,\text{years}$.

• Problem 2: Systolic Blood Pressure Sampling:

  • Scenario: Adult population mean systolic blood pressure $\mu = 120\,\text{mmHg}$, $\sigma = 20\,\text{mmHg}$. Simple random sample of $n = 100$ adults taken.
  • Question 1: How much does a typical adult's blood pressure differ from $120\,\text{mmHg}$?
  • Answer: $20\,\text{mmHg}$ (Population standard deviation $\sigma$).
  • Question 2: How much does a typical sample mean of size $n = 100$ differ from $120\,\text{mmHg}$?
  • Answer: $2\,\text{mmHg}$ ($SE = \frac{\sigma}{\sqrt{n}} = \frac{20}{\sqrt{100}} = \frac{20}{10} = 2\,\text{mmHg}$).

• Problem 3: UCLA vs. Second Graders Blood Sugar:

  • Data: UCLA students ($n_1 = 100$, $\bar{x}_1 = 5.5\,\text{mmol/L}$, $s_1 = 1.5$ ). Second graders ($n_2 = 80$, $\bar{x}_2 = 7.5\,\text{mmol/L}$, $s_2 = 2.2$ ).
  • Question A: Find a $95\%$ confidence interval for how much lower an average UCLA student's blood glucose level is than a second grader's ($\bar{x}_2 - \bar{x}_1$ ).
  • Observed Difference: $7.5 - 5.5 = 2.0$
  • $SE = \sqrt{\frac{1.5^2}{100} + \frac{2.2^2}{80}} = \sqrt{\frac{2.25}{100} + \frac{4.84}{80}} = \sqrt{0.0225 + 0.0605} = \sqrt{0.083} = 0.288$
  • $95\%$ CI: $2.0 \pm 1.96 \times 0.288 = 2.0 \pm 0.564 \rightarrow [1.436, 2.564]$
  • Question B: Is this difference statistically significant?
  • Answer: Yes, because the $95\%$ confidence interval does not contain zero.
  • Question C: Study classification?
  • Answer: Observational study. Explanatory variable is grade/education level (UCLA vs second grader).
  • Question D: Does attending UCLA cause blood sugar to drop?
  • Answer: No, age is a confounding variable; second graders consume more sugary food/candy.
  • Question E: The mean blood sugar level of ALL UCLA students is a:
  • Answer: Population parameter.
  • Question F: How much would another sample difference typically be off by?
  • Answer: $0.288$ (Standard Error of the difference).
  • Question G: How much does one UCLA student's blood sugar differ from the mean typically?
  • Answer: $1.5$ (Sample standard deviation of UCLA group).

Questions & Student Discussion

• Decimal Rounding on Exams:
  • Final numerical answers should be rounded to $3$ decimal places. Avoid rounding intermediate steps to prevent rounding errors.
• Significance Threshold ($\alpha$):
  • Default significance level is $\alpha = 0.05$ ($5\%$ ) unless explicitly specified otherwise in a problem.
• $p\text{-value}$ vs. $z\text{-score}$ Criteria:
  • Both statistics always yield consistent conclusions. If any ambiguity occurs, trust the $p\text{-value}$ as the primary decision metric.
• Exam Requirements:
  • No R coding syntax (e.g., pnorm, pt) is required to be executed or written during exams.