Comprehensive Biostatistics Notes: Hypothesis Testing, Error Analysis, Variance Tests, and Mean Comparisons
Foundations of Statistical Hypothesis Testing and Error Analysis
Conceptual Framework of Hypothesis Testing:
Statistical testing translates an initial observation or empirical question into two mutually exclusive mathematical assertions:
Null Hypothesis (): States that there is no true difference, effect, or relationship in the population (e.g., population means are equal: ; or the sample distribution conforms to a theoretical distribution).
Alternative Hypothesis / Working Hypothesis (): States that a genuine difference, effect, or deviation exists in the population (e.g., ).
Empirical data collected from samples are transformed into a single test statistic (such as , , , or ), which quantifies the deviation of the empirical observations from what is expected under .
A pre-defined significance level () is established prior to data evaluation (conventionally or ).
The -Value:
Verbatim Definition: The probability of obtaining a test statistic equal to or more extreme than the observed value, assuming that the null hypothesis () is strictly true.
Decision Rules:
If : The observed data are considered highly improbable under the assumption that is true. The null hypothesis is rejected in favor of (statistically significant result).
If : The observed data are compatible with random variation under . The null hypothesis is not rejected (fail to reject ).

Decision Matrix and Types of Statistical Errors:
Statistical decisions rely entirely on probabilities derived from sample data, creating an inherent risk of erroneous conclusions:
True State: is True:
Test Decision: Do not reject : Correct decision. Probability = .
Test Decision: Reject : Type I Error (Fehler 1. Art / False Positive). The test falsely detects an effect or difference when none exists in reality. Probability = .
True State: is False:
Test Decision: Do not reject : Type II Error (Fehler 2. Art / False Negative). The test fails to detect a genuine effect or difference. Probability = .
Test Decision: Reject : Correct decision. Probability = (known as the Statistical Power or Teststärke of the test).
Terminology Mapping:
False Positive: The test falsely indicates a positive response/effect; is rejected despite being true (corresponds exactly to a Type I error).
False Negative: The test falsely indicates no response/effect; is accepted/retained despite being false (corresponds exactly to a Type II error).
Real-World Consequences of Errors and the Base Rate Fallacy:
Tests do not provide absolute certainty. When conducting mass screening for rare events, the overwhelming majority of positive test results can be false positives even when test error rates appear very low.
Case Study 1: Schoolwide Scarlet Fever Screening ( Error Rates):
Population: Mass testing of first-grade students.
Prevalence: of students are infected ( sick children, healthy children).
Type I Error Rate (): ( specificity).
Type II Error Rate (): ( sensitivity / power).
Test Outcomes for Sick Cohort ( children):
True Positives: children test positive.
False Negatives: children test falsely negative.
Test Outcomes for Healthy Cohort ( children):
True Negatives: children test correctly negative.
False Positives: children test falsely positive.
Total Positive Tests: children test positive.
Positive Predictive Value (PPV):
Implication: Only of all individuals who test positive actually have scarlet fever; over of positive findings are false alarms.
Case Study 2: Scarlet Fever Screening with Highly Accurate Tests ( Error Rates):
Parameters: students; sick (), healthy (); , .
Sick children: test positive, tests false negative.
Healthy children: test negative, test false positive.
Total positive tests: .
False Positive Proportion: .
Implication: Even with an exceptional test accuracy of , more than two-thirds () of all positive test results remain false positives due to low base prevalence.
Case Study 3: COVID-19 Antigen Rapid Mass Testing:
Test parameters: Sensitivity = (), Specificity = ().
Scenario A (Low Prevalence: in inflected):
Of infected: True Positive, False Negative.
Of uninfected: False Positive (), True Negative.
Total positive results = . Only out of () are truly infected.
Scenario B (High Prevalence: in infected):
Of infected: True Positive, False Negative.
Of uninfected: False Positive (), True Negative.
Total positive results = . Truly infected positives = out of ().

Verification of Distributional Assumptions: Testing for Normality
Role of Normality Testing:
Parametric inferential procedures (including the -test, Student's -test, and Welch's -test) rely on the fundamental assumption that sample data originate from a normally distributed population.
Critical Prerequisite Rule: If either of two datasets being compared violates the assumption of a normal distribution, parametric tests (-test, Student's -test, Welch's -test) must not be applied.
Hypotheses for Normality Tests:
: The sample distribution does not differ significantly from a normal distribution ().
: The sample distribution differs significantly from a normal distribution ().
Decision Criteria:
: Fail to reject . Data are assumed to be normally distributed.
: Reject . Data are significantly non-normal.
Methods for Assessing Normality:
Visual Assessment Methods:
Histogram: Allows rapid visual inspection for symmetry, unimodality, and pronounced skewness or heavy tails.
Quantile-Quantile Plot (Q-Q Plot): Plots empirical sample quantiles against theoretical normal distribution quantiles. If the data are normally distributed, the data points fall directly along the linear reference line. S-shaped curves, systematic arcs, or deviations at the extremes reveal skewness and kurtosis.
Inferential Testing Methods:
Kolmogorov-Smirnov Test (KS-Test): Compares the cumulative empirical distribution function of the sample against the cumulative distribution function of a theoretical normal distribution. Measures maximum distance .
Shapiro-Wilk Test: Evaluates the correlation between sample order statistics and expected normal values, comparing variance components to compute test statistic . Highly recommended for small to moderate sample sizes.


Contradictions Among Testing Procedures:
Statistical tests for normality can yield conflicting results when applied to identical datasets, demonstrating why visual validation must accompany analytical testing:
One-Sample Kolmogorov-Smirnov Test on
mydata:Statistic:
-value =
Formal Conclusion: , failing to reject . The test indicates that the data distribution does not deviate significantly from a normal distribution.
Shapiro-Wilk Normality Test on
mydata:Statistic:
-value =
Formal Conclusion: , soundly rejecting . The test indicates that the data deviate significantly from normality.
Resolution via Visual Inspection: The normal Q-Q plot of
mydataclearly displays substantial non-linear tail deviations, and the histogram demonstrates pronounced right-skewness. Despite the non-significant KS-test result, the dataset is objectively non-normal, confirming the Shapiro-Wilk test result.
Standards for Rigorous Reporting of Results:
A complete and scientific presentation of test results must include:
A precise verbal description of the biological or empirical outcome.
The exact formal test applied.
The numerical test statistic (, , , , etc.) including associated degrees of freedom ().
The exact -value reported with precision (avoiding vague thresholds such as or unless values are extremely small, e.g., ).
Reporting Examples:
Significant Non-Normality: The distribution of the data deviates significantly from a normal distribution (Shapiro-Wilk test, , ).
Non-Significant Deviation: The distribution of the data does not deviate significantly from a normal distribution (KS-test, , ).
Visual Finding: Visual inspection using a Q-Q plot revealed that the data are not normally distributed.
Standards for Graphical Figures: All axes must have descriptive text labels including measurement units (e.g., parts per hundred million [], grams []). Formal publications and theses require legends and standalone figure captions.
Testing for Equality of Variances (Homoscedasticity)
Overview and Significance:
Homoscedasticity refers to the assumption that two or more compared groups exhibit equal population variances ().
Evaluating variance equality determines whether standard parametric comparisons (such as Student's -test) are legitimate, or whether adjustments (Welch's -test) or non-parametric alternatives must be employed.

Empirical Dataset: Ozone Concentrations in Nurseries:
Study design: Three commercial garden centers (
gartenA,gartenB,gartenC) measured ozone concentrations across distinct time points each ( total observations). Ozone is recorded in parts per hundred million ().R Data Ingestion and Summary Commands:
R gaerten <- read.table("05_gaerten.txt", header = TRUE) tapply(gaertenstandort, var) tapply(gaertenstandort, mean) Summary Statistics:
Garden A: Mean = , Variance =
Garden B: Mean = , Variance =
Garden C: Mean = , Variance =
Empirical Comparison: Garden A and Garden C possess nearly identical means ( vs ), but the variance of Garden A is more than three times higher than that of Garden C ( vs ).
Fisher's F-Test for Two Variances (Parametric):
Strict Requirement: Both datasets must be normally distributed.
Hypotheses:
: (variance ratio equals \,: ).
: (variance ratio differs from \,: ).
Test Statistic Calculation:
For Garden A versus Garden C:
Interpretation and Directional Symmetry:
If the ratio is inverted (), the resulting statistical probability is completely identical.
Cumulative probability with : .
Upper-tail probability: .
Lower-tail probability for inverted ratio: .
Two-sided -value: .

Influence of Degrees of Freedom on the F-Distribution:
Degrees of freedom for each group are determined by sample size: and .
Sample Size Effect: With increasing sample size (), the cumulative -distribution curve becomes markedly steeper around .
With few data points (e.g., ), variance estimates are uncertain; an -value of yields a cumulative probability of approximately (), failing to reach significance.
With large sample sizes (e.g., ), variance estimates are highly accurate; an -value of yields a cumulative probability of approximately (), which is statistically significant.
Principle: As sample size increases, smaller deviations of the variance ratio from are required to reject the null hypothesis of equal variances.

Step-by-Step F-Test Implementation in R:
Data loading and subgroup creation:
R gaerten <- read.table("05_gaerten.txt", header = TRUE) gA <- subset(x = gaerten, standort == "gartenA") gB <- subset(x = gaerten, standort == "gartenB") gC <- subset(x = gaerten, standort == "gartenC")
Normality confirmation via Shapiro-Wilk test:
R shapiro.test(gAozon) # W = 0.9879, p = 0.9941 shapiro.test(gBozon) shapiro.test(gC$ozon) All -values are greater than ; normality assumption is satisfied.Execution of the variance test:
R var.test(gAozon) R Output Details:
,
-value =
Confidence Interval:
Sample estimate (ratio of variances):
Formal Results Presentation:
Garden A vs Garden C: The variances of the ozone values measured in Garden A and Garden C differ significantly (-test, , ).
Garden A vs Garden B: The variances of the ozone values measured in Garden A and Garden B differ significantly (-test, , ).
Garden B vs Garden C: The variances of the ozone values measured in Garden B and Garden C do not differ significantly (-test, , ).
Levene's Test for Homogeneity of Variances (Non-Parametric / Robust):
Indications: Used when data deviate from a normal distribution. While the -test possesses greater statistical power for purely normal data, it is sensitive to violations of normality. Levene's test provides a robust assessment of homoscedasticity across two or more groups ().
Mathematical Formulation:
Calculate the absolute deviation of each data point from its group mean (or group median):
Compute the average deviation within each individual group :
Compute the grand mean of absolute deviations across all observations: where .
Calculate the test statistic , which approximately follows an -distribution:
Degrees of Freedom:
Numerator degrees of freedom:
Denominator degrees of freedom:
Median-Centered Levene Test (Brown-Forsythe): Modern implementations replace group means with group medians () to enhance robustness against extreme outliers.
R Implementation via the
carPackage:R install.packages("car") library("car") gAC <- subset(gaerten, standort == "gartenA" | standort == "gartenC") leveneTest(gACstandort) R Output Details:
Center:
median,
Formal Presentation of Levene Result: The variances of the ozone values measured in Garden A and Garden C differ significantly (Levene's test, , ).
Excursus: Logical Operators in R Data Subsetting:
The
|operator represents logical OR (at least one condition must hold true).The
&operator represents logical AND (both conditions must simultaneously hold true).Behavioral Analysis:
standort == "gartenA" & standort == "gartenC": Produces an empty subset (empty set) because no row can contain both character strings in the same column.standort == "gartenA" | ozon == 2.06: Extracts every row belonging to Garden A plus entry 21 (which has an ozone level of ).standort == "gartenB" & ozon == 2.06: Extracts exclusively entry 21 (which simultaneously satisfies both conditions).
Testing Differences Between Two Independent Means
Methodological Decision Tree for Group Mean Comparisons:
Assess Normality: Check both datasets using KS-tests, Shapiro-Wilk tests, or Q-Q plots.
If normal: proceed to step 2.
If non-normal: proceed directly to non-parametric testing via the Wilcoxon Rank-Sum Test (Mann-Whitney U-Test).
Assess Homoscedasticity (Equal Variance):
Test using the parametric -test (or robust Levene-test).
If variances are equal (): proceed to Student's -test.
If variances are unequal (): proceed to Welch's -test.
Student's Two-Sample t-Test (Parametric, Homoscedastic):
Historical Context: Developed by William Sealy Gosset (1876–1937), an Oxford-trained mathematician and chemist employed by the Guinness Brewery in Dublin. Because corporate policy prohibited staff from publishing proprietary brewing discoveries, Gosset published the distribution under the pseudonym "Student" in 1908.

Research Case Study: Blue Tit Body Mass: Empirical observation indicates geographical variation in male blue tit (Cyanistes caeruleus) body mass between Hamburg and Berlin.
Hamburg Sample: , mean mass
Berlin Sample: , mean mass
Hypotheses:
: (mean body mass does not differ significantly between cities).
: (mean body mass differs significantly between cities).
Mathematical Definition of the t-Statistic: where represents the pooled common standard deviation of both samples, and the denominator represents the standard error of the difference between two sample means.

Component Breakdown:
Numerator: Observed difference in sample means ().
Denominator: Combined standard error:
Calculated -statistic:
Degrees of Freedom:

One-Sided versus Two-Sided Testing:
Two-Sided (Nondirectional) Test: Applied when no prior theoretical or physiological basis predicts which mean will be larger. Critical rejection areas are placed in both tails ( and ).
One-Sided (Directional) Test: Applied when explicit prior knowledge dictates that an effect can occur in only one direction (e.g., comparing running speeds of hares versus snails; biological constraints preclude snails from being faster). The entire rejection area () is concentrated in a single tail.
Execution in R:
R meisen <- read.table("05_Blaumeisen.txt", header = TRUE) m_hh <- subset(meisen, ort == "hamburg") m_b <- subset(meisen, ort == "berlin") t.test(m_hhgewicht, var.equal = TRUE) R Output Results:
, , -value =
Confidence Interval:
Mean of (Hamburg) = , Mean of (Berlin) =
Formal Presentation of t-Test Result: The means of the blue tit body weights in Hamburg () and Berlin () do not differ significantly (-test, , , ).
Welch's t-Test (Parametric, Heteroscedastic):
Purpose: Used when sample distributions are normal but group variances are unequal ().
Mathematical Adjustment: Discards the pooled standard deviation in favor of separate group sample variances:
Degrees of Freedom: Corrected using the Welch-Satterthwaite equation, which typically yields a non-integer, reduced degrees of freedom.
Execution in R: Implemented by specifying
var.equal = FALSE(which is the default in R):R t.test(m_hhgewicht, var.equal = FALSE) R Output Results: , , -value = . Because the tit dataset exhibited equal variances, the Welch correction yields an identical conclusion with a minor adjustment to degrees of freedom ( vs ).
Wilcoxon Rank-Sum Test / Mann-Whitney U-Test (Non-Parametric):
Indications: Non-parametric equivalent of the independent two-sample -test. Applied when data violate normality assumptions.
Core Principle: Evaluates the relative ranks of data points rather than their raw metric values. Focuses on ordinal positions when all pooled observations are arranged in ascending order.
Step-by-Step Calculation Procedure:
Pool both independent samples () containing and observations into a combined dataset of size .
Sort all values in ascending order, retaining the group identity of each observation.
Assign ranks from (smallest value) to (largest value). For tied values, assign the average of the ranks they occupy.
Calculate the rank sum for each group:
Compute the Wilcoxon test statistic for each sample:
The smaller of the computed values is compared against critical values tabulated for sample sizes and .
Execution in R:
R wilcox.test(m_hhgewicht)
R Output Results: , -value = .
Formal Presentation of Wilcoxon Result: The means of the blue tit body weights in Hamburg () and Berlin () do not differ significantly (Wilcoxon rank-sum test, , ).
Analysis of Paired Samples and Advanced Median Comparison
Principles of Paired (Dependent) Experimental Designs:
Definition: A paired design occurs when each data point in sample A is uniquely and non-randomly linked to a specific data point in sample B (e.g., repeated measures on the same individual, split-plot agricultural experiments, or spatial upstream/downstream matching).
Conceptual Difference:
Unpaired Test: Compares overall group means (ignoring individual correspondences). High background variance between experimental units can obscure genuine differences.
Paired Test: Subtracts each paired observation to create a single vector of paired differences (), testing whether the mean difference () differs significantly from zero.
Case Study: Aquatic Invertebrate Diversity Around Wastewater Plants:
Study System: Abundance of aquatic macroinvertebrates recorded across different rivers at two sampling stations per river: upstream (
up, before plant) and downstream (down, after plant).Data Characteristics: River-to-river biodiversity varies dramatically across natural landscapes (some rivers support only to species, while others support to ). However, downstream counts are systematically and consistently higher than upstream counts within nearly every river.


Paired Student's t-Test:
Mathematical Formula: where is the mean of the paired differences, is the sample standard deviation of the paired differences, and is the number of paired differences.
Comparing Statistical Outcomes in R:
Incorrect Unpaired Test:
R t.test(streamsdown, var.equal = TRUE) # Output: p-value = 0.6231 Conclusion: No statistically significant difference. The substantial variance among different rivers masks the local treatment effect.
Correct Paired Test:
R t.test(streamsdown, paired = TRUE, var.equal = TRUE) # Output: p-value = 3.162e-07 Conclusion: Highly statistically significant difference (). Accounting for river-specific pairings eliminates inter-river variance and detects the true biological impact of the facility.Paired Wilcoxon Signed-Rank Test (Non-Parametric):
Indications: Used for paired/matched samples when the differences () violate normality.
Calculation Procedure:
Compute the raw differences for every matched pair: .
Take the absolute values of these differences: .
Rank the absolute differences from lowest to highest: .
Calculate separate rank sums for pairs with positive versus negative original differences:
The test statistic is based on the minimum rank sum ().
Comparing Statistical Outcomes in R:
Unpaired Wilcoxon Test:
R wilcox.test(streamsdown) # Output: p-value = 0.6205
Paired Wilcoxon Signed-Rank Test:
R wilcox.test(streamsdown, paired = TRUE) # Output: p-value = 5.722e-06 * Conclusion: The paired Wilcoxon test detects a significant effect (), whereas the unpaired version fails entirely ().Visual Median Comparison via Notched Boxplots:
Methodology: Adding notches to standard boxplots provides a graphical test for median differences:
R boxplot(data = meisen, gewicht ~ ort, notch = TRUE, xlab = "Stadt", ylab = "Gewicht (g)")
Interpretation Rules:
The notch width illustrates an approximate confidence interval around the sample median.
Overlapping Notches: Strong visual indicator that population medians do not differ significantly (e.g., comparing Berlin and Hamburg tit weights).
Non-Overlapping Notches: Strong visual indicator that population medians differ significantly (e.g., comparing Turku tit weights to Berlin or Hamburg).

Summary of Biostatistical Testing Procedures
Comprehensive Testing Matrix:
Assessing Normality (One Sample vs Theoretical Normal):
Quantitative Tests: Shapiro-Wilk test (preferred for smaller ), Kolmogorov-Smirnov test.
Visual Assessment: Normal Q-Q plots (quantiles vs theoretical line), histograms.
Assessing Homoscedasticity (Equality of Variances):
Normal Data: Fisher's -test (
var.test()). Ratio of variances .Non-Normal Data: Levene's test (
leveneTest()in librarycar). Compares group absolute deviations to overall absolute deviations.
Comparing Means / Medians (Two Groups):
Normal Data + Equal Variances (Independent): Student's Two-Sample -test (
t.test(..., var.equal = TRUE)). Compares difference between group means to the pooled standard error.Normal Data + Unequal Variances (Independent): Welch's -test (
t.test(..., var.equal = FALSE)). Employs unpooled standard errors and adjusted degrees of freedom.Non-Normal Data (Independent): Wilcoxon Rank-Sum Test / Mann-Whitney U-Test (
wilcox.test()). Transforms observations into ranks and sums ranks per group.Normal Data (Paired/Matched): Paired Student's -test (
t.test(..., paired = TRUE)). Analyzes the vector of pairwise differences against zero.Non-Normal Data (Paired/Matched): Paired Wilcoxon Signed-Rank Test (
wilcox.test(..., paired = TRUE)). Analyzes ranked absolute paired differences separated by sign ().Visual Assessment of Medians: Notched boxplots (
boxplot(..., notch = TRUE)). Evaluates confidence intervals of medians via notch overlap.