l19-Comparing Continuous Variables: Paired T-Tests and Nonparametric Alternatives

Strategies for Statistical Success and Exam Preparation

• Creating an Equation List: The most effective study strategy for quantitative analysis is to identify and compile the most important equations for each type of statistical test.

• Adding Context: Notes should be added to each equation to explain its application, variables, and context.

• The "One-Page" Cheat Sheet: This compiled list should serve as the foundation for the single page of notes allowed during the final exam.

• Iterative Refinement: Students are encouraged to start this document early, specifically while processing lecture notes and labs, to refine it throughout the study period.

Learning Objectives

• Perform and interpret two-sample t-tests.

• Perform and interpret paired t-tests.

• Understand the criteria for choosing nonparametric tests over parametric t-tests.

• Develop proficiency in R for calculating and performing statistical analyses.

Revisit: The Two-Sample T-Test for Independent Samples

• Definition: A method used to compare a continuous variable between two independent groups or samples.

• Core Assumptions:

  • The data is continuous.

  • The data follows a normal distribution.

  • The default assumption is equal variances between the two samples.

• Test Statistic Formulas:

  • Traditional formula using pooled standard deviation ($s_p$):

    • $t = \frac{\bar{x}_1 - \bar{x}_2}{s_p \times \sqrt{\frac{1}{n_1} + \frac{1}{n_2}}}$

  • Alternative formulation based on standard error of the difference:

    • $Standard Error (SE) = \sqrt{\frac{V_1}{n_1} + \frac{V_2}{n_2}}$

    • Where $V_1$ and $V_2$ are the variances of sample 1 and sample 2 respectively.

    • This alternative is often easier to calculate using provided summary statistics.

Independent vs. Paired Comparisons

• Unpaired (Independent) Comparisons: Measurements are taken from entirely separate individuals in each group.

  • Example: Testing two different toothpastes on two separate groups of people (Group A gets regular, Group B gets experimental).

• Paired (Non-Independent) Comparisons: Measurements are repeated on the same individual or unit.

  • Example: Measuring the same individual's results using one toothpaste and then the other after a rotation.

• The Power of Pairing: Pairing is statistically powerful because it eliminates variation caused by differences between individuals (e.g., genetic predispositions for tooth decay), allowing the test to isolate the effect of the treatment.

The Paired T-Test

• Purpose: Compares the average difference between two paired groups to a null hypothesis value of zero.

• Comparison to One-Sample T-Test: A paired t-test is essentially a one-sample t-test where the sample consists of the calculated differences ($d_i = x_{1,i} - x_{2,i}$) between pairs.

• Assumptions:

  • The dependent variable ($y$) must be continuous.

  • The differences between the pairs must be normally distributed.

  • The sample of pairs must be a random sample from the population.

• Mathematics of the Test Statistic:

  • $t = \frac{\bar{d} - 0}{SE_{\bar{d}}}$

  • Here, $\bar{d}$ is the mean of all differences, and $SE_{\bar{d}}$ is the standard error of those differences.

• Efficiency: The test subtracts out covariance, making it more likely to detect true differences between means than an unpaired test.

• Constraint: A paired t-test cannot be performed on independent samples; the experimental design must be intrinsically paired.

Case Study: Snapper Populations in Marine Reserves (2004 vs. 2011)

• Study Design: Researchers measured the size of Snapper fish in the same eight marine reserves at two different time points (2004 and 2011).

• Research Question: Do the 2011 samples have a higher average number of snapper above the legal catch size compared to 2004?

• Hypothesis Type: This is a one-tailed hypothesis test because an increase is specifically expected over time.

• Data Management in R:

  • File: snapper_reserves.csv.

  • Missing values were present in the data, requiring removal (e.g., sample size dropping from 8 to 7 after na.omit or specific row exclusion).

  • Visualization: A box plot comparing the two columns (2004 vs. 2011) showed a visually apparent increase in legal-sized fish.

• Step-by-Step R Calculation:

  • Calculate differences: diff <- snapper_reserves$legal_2004 - snapper_reserves$legal_2011.

  • Calculate mean of differences: mean(diff).

  • Calculate standard error: sd(diff) / sqrt(n_row(snapper_reserves)).

  • Calculate t-statistic: mean_diff / se_diff.

  • Determine p-value: Use pt(t_statistic, df).

• Degrees of Freedom ($df$): For paired tests, $df = n - 1$, where $n$ is the number of pairs (not the total number of observations).

Normality and Q-Q Plot Interpretation

• Normal Distribution: The Q-Q plot shows points following a straight diagonal line.

• Poisson Distribution: A highly skewed sample with many small values results in a curved Q-Q plot.

• Right-Skewed Distribution: A distribution where the right tail is larger than the left produces a "hockey stick" shape in the Q-Q plot.

• Data Transformation: Transformations (like log transforms) are used to make Q-Q plots straighter, though some distributions remain skewed regardless of transformation.

Nonparametric Alternatives: The Wilcoxon Tests

• Definition: Parameter-free tests used when data is not normally distributed and cannot be transformed effectively, or when many outliers are present.

• Test Names:

  • Wilcoxon Rank Sum Test: Used for two independent samples (also known as the Mann-Whitney U Test).

  • Wilcoxon Signed Rank Test: Used for paired samples.

• Mechanism: The Rank Transformation:

  1. Combine all samples into one array.

  2. Sort values from lowest to highest.

  3. Assign a rank to each (ties receive the average rank).

  4. Sum the ranks for each group and compare the average ranks.

• Null Hypothesis ($H_0$): The average rank within the two samples is equal.

• Underlying Distribution: Nonparametric tests assume an underlying uniform distribution because every rank should occur equally.

• Trade-offs:

  • Pros: Handles outliers well; no normality assumption.

  • Cons: Cannot be "back-transformed"; more conservative than t-tests (generally results in higher p-values); more likely to fail to reject the null hypothesis when a t-test might suggest significance.

Statistical Functions in R

• T-Test Function: t.test(sample1, sample2, paired = TRUE, alternative = "less").

  • Specify paired = TRUE for paired data.

  • Specify alternative = "less" or "greater" for one-tailed tests.

• Wilcoxon function: wilcox.test(sample1, sample2, paired = TRUE, alternative = "less").

  • R may provide warnings about ties and zeros; in this specific context, these can often be ignored as R uses an alternative approach to calculate the p-value.

Summary of Equations and Degrees of Freedom

• Paired T-Test:

  • Test Statistic: $t = \frac{\bar{d}}{SE_{\bar{d}}}$

  • Degrees of Freedom: $df = n - 1$ (where $n$ is the number of pairs).

• Two-Sample T-Test (Independent):

  • Test Statistic: $t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}$

  • Degrees of Freedom: More complex; often approximated by R (e.g., Welch's t-test).

Practice Scenarios and Audience Discussion

• Scenario 1 (Plastic Ban): Researchers counted plastic on 10 Canterbury beaches before and after a ban.

  • Correct Test: Paired t-test, one-tailed (testing specifically for a decrease).

• Scenario 2 (Interpreting Output): Comparing snapper size in 2004 vs. 2011.

  • Question: Why does R say "alternative hypothesis: true mean difference is less than zero"?

  • Answer: This is a restatement of how the user specified the test (alternative = "less"), indicating a one-tailed test in the lower tail. It is not the result, but the framework. If the first year (2004) minus the second year (2011) results in a negative mean difference (e.g., $-8.14$), it signifies that the 2011 size was larger.