Comprehensive Guide to the t Distribution, Hypothesis Testing, and Confidence Intervals

Foundations of Hypothesis Testing: Z Test Versus Single-Sample t Test

• Comparison of statistical tests based on population parameter knowledge:

  • In a $Z$ test, a sample mean ($M$) is compared against a known population mean ($\mu$) when the population standard deviation ($\sigma$) is known.

  • In a single-sample $t$ test, a sample mean ($M$) is compared against a population with a known mean ($\mu$), but the population standard deviation ($\sigma$) is unknown.

  • When the true population standard deviation ($\sigma$) is unknown, it must be estimated using the sample standard deviation ($s$).

• General conceptual structure of the $t$ statistic ratio:

  • The $t$ statistic reflects a fundamental ratio across all inferential designs:     $\text{General } t = \frac{\text{Sample Statistic} - \text{Hypothesized Parameter}}{\text{Estimated Standard Error}}$

  • Concretely expressed as:     $\text{Conceptual } t = \frac{\text{what you got in the sample} - \text{what you'd expect to get if } H_0 \text{ is true}}{\text{how much of a difference you'd expect because you're sampling}}$

• Formula for the single-sample $t$ test:   $t = \frac{M - \mu_M}{s_M}$

  • $M$ represents the sample mean obtained from empirical observations.

  • $\mu_M$ represents the population mean of the distribution of sample means, which equals the known population mean ($\mu$).

  • $s_M$ represents the estimated standard error of the mean, calculated as:     $s_M = \frac{s}{\sqrt{n}}$

Degrees of Freedom and Sample Variance

• Price paid for estimating parameters:

  • Estimating an unknown population parameter ($\sigma$) using sample data ($s$) introduces mathematical restrictions, creating a penalty captured by degrees of freedom ($df$).

• Formal definitions of degrees of freedom ($df$):

  • Physical/mechanical definition: The number of independent ways by which a dynamic system can move without violating any constraint imposed upon it.

  • Mathematical/statistical definition: A mathematical restriction that needs to be put in place when estimating one statistic ($\sigma \rightarrow s$) from an estimate of another ($\mu \rightarrow M$).

• Sample standard deviation formula incorporating degrees of freedom:   $s = \sqrt{\frac{\sum (X - M)^2}{n - 1}}$

  • The sample mean ($M$) serves as an estimate of the population mean ($\mu$).

  • The resulting sample standard deviation ($s$) serves as an estimate of the population standard deviation ($\sigma$).

  • The calculation of degrees of freedom for a single sample is:     $df = n - 1$

Properties and Characteristics of the t Distribution

• Source of variability in the $t$ distribution:

  • In a $t$ test, $\sigma$ is estimated with $s$.

  • From sample to sample, the numerical value of $s$ fluctuates due to sampling error.

  • This additional source of error results in substantially greater variability within the $t$ distribution compared to the standard normal ($Z$) distribution.

• Structural and morphological comparison between $Z$ and $t$ distributions:

  • Thicker tails: The $t$ distribution exhibits heavier, thicker tails and a flatter peak than the normal curve, accounting for a higher probability of extreme scores.

  • Critical value differences: At an identical significance threshold (e.g., $\alpha = .05$), the critical value of $t$ ($t_{\text{crit}}$) lies farther out into the tails than the critical value of $Z$ ($Z_{\text{crit}}$).

  • Rejection difficulty: It is harder to reject the null hypothesis with a $t$ test than with a $Z$ test, requiring a more extreme sample mean (one located farther from zero) to achieve statistical significance.

  • Effect of sample size and $df$:

  • The $t$ distribution dynamically changes shape depending on the degrees of freedom ($n - 1$).

  • When sample size ($n$) is small, degrees of freedom are low, producing the thickest tails and most dispersion.

  • As sample size ($n$) approaches infinity ($n \rightarrow \infty$), degrees of freedom approach infinity ($df = \infty$), and the $t$ distribution becomes identical to the standard normal distribution ($t = Z$).

Critical Values and Table of the t Distribution

• Table layout and reading conventions:

  • Critical values in Table B.2 are indexed by degrees of freedom ($df$) along the rows and alpha levels ($\alpha$) along the columns.

  • Proportions are specified either for one tail (either right or left) or for two tails combined.

• Selected critical values of the $t$ distribution across degrees of freedom:

  • At $df = 1$:

  • One tail: $0.25 \rightarrow 1.000$, $0.10 \rightarrow 3.078$, $0.05 \rightarrow 6.314$, $0.025 \rightarrow 12.706$, $0.01 \rightarrow 31.821$, $0.005 \rightarrow 63.657$

  • Two tails: $0.50 \rightarrow 1.000$, $0.20 \rightarrow 3.078$, $0.10 \rightarrow 6.314$, $0.05 \rightarrow 12.706$, $0.02 \ \rightarrow 31.821$, $0.01 \rightarrow 63.657$

  • At $df = 2$:

  • Two tails: $0.05 \rightarrow 4.303$, $0.01 \rightarrow 9.925$

  • At $df = 3$:

  • Two tails: $0.05 \rightarrow 3.182$, $0.01 \rightarrow 5.841$

  • At $df = 4$:

  • Two tails: $0.05 \rightarrow 2.776$, $0.01 \rightarrow 4.604$

  • At $df = 5$:

  • Two tails: $0.05 \rightarrow 2.571$, $0.01 \rightarrow 4.032$

  • At $df = 6$:

  • Two tails: $0.05 \rightarrow 2.447$, $0.01 \rightarrow 3.707$

  • At $df = 7$:

  • Two tails: $0.05 \rightarrow 2.365$, $0.01 \rightarrow 3.499$

  • At $df = 8$:

  • Two tails: $0.05 \rightarrow 2.306$, $0.01 \rightarrow 3.355$

  • At $df = 9$:

  • Two tails: $0.05 \rightarrow 2.262$, $0.01 \rightarrow 3.250$

  • At $df = 10$:

  • Two tails: $0.05 \rightarrow 2.228$, $0.01 \rightarrow 3.169$

  • At $df = 20$:

  • Two tails: $0.05 \rightarrow 2.086$, $0.01 \rightarrow 2.845$

  • At $df = 30$:

  • Two tails: $0.05 \rightarrow 2.042$, $0.01 \rightarrow 2.750$

  • At $df = 60$:

  • Two tails: $0.05 \rightarrow 2.000$, $0.01 \rightarrow 2.660$

  • At $df = 120$:

  • Two tails: $0.05 \rightarrow 1.980$, $0.01 \rightarrow 2.617$

  • At $df = \infty$ (equivalent to $Z$):

  • One tail: $0.05 \rightarrow 1.645$, $0.025 \rightarrow 1.960$, $0.01 \rightarrow 2.326$, $0.005 \rightarrow 2.576$

  • Two tails: $0.10 \rightarrow 1.645$, $0.05 \rightarrow 1.960$, $0.02 \rightarrow 2.326$, $0.01 \rightarrow 2.576$

Single-Sample t Test Applications and Empirical Studies

• Harlow (1959) Infant Attachment Study:

  • Experimental arrangement: Infant monkeys were placed in cages with two artificial surrogate mothers:

  • A bare wire mesh mother equipped with a feeding nursing bottle.

  • A soft terry cloth mother providing no access to food.

  • Baseline and null model: In a typical day, infant monkeys spend an aggregate of $18\text{ hours}$ clinging to a mother surrogate. If there were no preference, clinging time would divide equally, yielding a population baseline mean of $\mu = 9\text{ hours}$.

  • Empirical observation: Typical monkeys spent roughly $15\text{ hours per day}$ clinging to the terry cloth mother, evidencing strong attachment to contact comfort.

  • Sample parameters:

  • Sample size: $n = 10$

  • Sample mean: $M = 15.3\text{ hours per day}$

  • Sample standard deviation: $s = 5.2\text{ hours}$

  • Hypothesized mean: $\mu = 9\text{ hours}$

  • Significance criterion: $\alpha = .05$

  • Test setup:

  • $df = n - 1 = 10 - 1 = 9$

  • Evaluate if monkeys spend significantly more time with the softer mother than expected by chance.

• COVID-19 Quarantine and CDC Exercise Recommendations:

  • Baseline guideline: The CDC recommends that adults engage in $150\text{ minutes}$ of exercise per week, represented as $30\text{ minutes a day}$ across $5\text{ days}$ ($\mu = 5$).

  • Research question: Determine if pandemic conditions (closed fitness facilities, shelter-in-place orders) significantly altered exercise adherence relative to the CDC guideline of $5\text{ days per week}$.

  • Sample data ($n = 9$ adults recorded days exercised per week):     $\{5, 6, 4, 3, 4, 1, 0, 5, 4\}$

  • Test parameters:

  • Population benchmark: $\mu = 5$

  • Significance level: $\alpha = .01$

  • Degrees of freedom: $df = 9 - 1 = 8$

Estimation via Confidence Intervals

• Hypothesis testing versus parameter estimation:

  • If the analytical goal is strictly to identify whether an effect or difference exists, conduct a hypothesis test.

  • If the analytical goal is to quantify the size or magnitude of an effect, construct a confidence interval.

  • Confidence intervals are particularly informative following null hypothesis rejection to define the probable scale of the difference.

• Formula for the single-sample $t$ confidence interval:   $M \pm (t_{\text{conf}})(s_M)$

• Properties of the critical multiplier ($t_{\text{conf}}$):

  • Extracted directly from the $t$ distribution critical value table.

  • Located via degrees of freedom ($df$) and the designated error rate ($\alpha$).

  • Rule of symmetry: $t_{\text{conf}}$ is always two-tailed, even when the preceding hypothesis test was directional (one-tailed).

Precision, Sample Size, and Interpretation of Confidence Intervals

• Confidence levels and margin of error:

  • A $95\%$ confidence interval corresponds to an error threshold of $\alpha = .05$.

  • A $99\%$ confidence interval corresponds to an error threshold of $\alpha = .01$.

  • Width comparison: A $99\%$ confidence interval is wider and encompasses more values than a $95\%$ interval.

  • Precision comparison: A $95\%$ confidence interval is narrower and therefore more precise than a $99\%$ interval.

• Strategies to increase confidence interval precision:

  • Lower the confidence coefficient: Moving from a $99\%$ confidence interval to a $95\%$ confidence interval reduces the width.

  • Increase sample size ($n$):

  • Standard error varies inversely with sample size: $s_M = \frac{s}{\sqrt{n}}$.

  • Increasing $n$ shrinks $s_M$.

  • Larger samples mitigate the disproportionate leverage of single outlier scores on $M$ , yielding narrower, more precise intervals.

• Theoretical interpretation and the "ring toss" metaphor:

  • True parameter fixedness: The population mean ($\mu$) is a fixed, unchanging quantity; it does not move.

  • Intervals as moving rings: Across repeated independent sampling, each generated confidence interval varies in position depending on the sample mean.

  • Ring toss imagery:

  • The target peg represents the true population mean ($\mu$).

  • The thrown rings represent confidence intervals constructed from individual samples.

  • In a $95\%$ interval system, $95\%$ of all constructed intervals successfully enclose the target peg ($\mu$), whereas $5\%$ miss.

  • A $99\%$ interval uses a wider ring, capturing the peg in $99\%$ of tosses.

Dependent Samples t Test: Repeated Measures and Matched Pairs

• General features of the dependent samples design:

  • Eliminates the need to know population parameters ($\mu$ or $\sigma$).

  • Compares two sets of scores that are linked or paired directly with one another.

• Core design classifications:

  • Repeated measures (within-subjects): Measuring the identical subject across multiple time points or experimental conditions (e.g., "before" treatment versus "after" treatment).

  • Related measures (matched subjects): Pairing distinct individuals one-to-one on key extraneous variables that could affect the dependent variable.

• Test statistic formula:   $t = \frac{M_D - \mu_{M_D}}{s_{M_D}}$

  • $M_D$ is the sample mean of the computed difference scores ($D = X_1 - X_2$).

  • $\mu_{M_D}$ represents the hypothesized mean difference under the null hypothesis, which is always $0$:     $\mu_{M_D} = 0$

  • $s_{M_D}$ is the estimated standard error of the difference scores, calculated as:     $s_{M_D} = \frac{s_D}{\sqrt{n}}$

  • Confidence interval formula for dependent samples:     $M_D \pm (t_{\text{conf}})(s_{M_D})$

• Statistical power and individual difference variability:

  • Within-subjects designs are statistically more powerful than between-subjects designs.

  • Increased power results from reducing individual difference variability by measuring the same individual repeatedly.

  • Location in the formula: Individual differences reside in the denominator of the $t$ test (the standard error term); eliminating this variance decreases standard error and inflates the observed $t$ value.

• Methodological limitations of within-subjects designs:

  • Order effects: The experience gained during an initial trial or condition can systematically distort participant behavior or performance in subsequent trials.

  • Practice effects: Incremental experience, task familiarity, or fatigue accumulated across successive trials can alter performance independently of the treatment.

Dependent Samples t Test Empirical Studies

• Pelton (1983) Olympic Marksmen Study:

  • Experimental premise: Olympic marksmen exhibit enhanced shooting accuracy when timing trigger pulls to occur between heartbeats rather than during heartbeats, avoiding fine mechanical vibrations.

  • Sample details: $n = 6$ Olympic marksmen firing precision rounds during both phases; accuracy scores recorded on a scale out of 100.

  • Paired observations:

  • Marksman 1: During = 93, Between = 98

  • Marksman 2: During = 90, Between = 94

  • Marksman 3: During = 95, Between = 96

  • Marksman 4: During = 92, Between = 91

  • Marksman 5: During = 95, Between = 97

  • Marksman 6: During = 91, Between = 97

  • Hypothesis parameters: Two-tailed or directional evaluation with $\alpha = .05$. If significant, construct a $95\%$ confidence interval for the population mean difference score ($\mu_D$).

• Pet Ownership and Mood Inventory Study:

  • Experimental premise: Evaluating whether single individuals who own pets differ in subjective mood from single non-pet owners.

  • Control matching: Non-pet owners are matched one-to-one with pet owners on income, social support network, and general physical health.

  • Sample data ($n = 6$ matched pairs; mood inventory scores):

  • Pair 1: Non-Pet Owner = 10, Pet Owner = 15

  • Pair 2: Non-Pet Owner = 9, Pet Owner = 9

  • Pair 3: Non-Pet Owner = 11, Pet Owner = 10

  • Pair 4: Non-Pet Owner = 11, Pet Owner = 19

  • Pair 5: Non-Pet Owner = 5, Pet Owner = 17

  • Pair 6: Non-Pet Owner = 9, Pet Owner = 15

  • Hypothesis parameters: Significance level set at $\alpha = .01$ with $df = n - 1 = 5$.

Independent Samples t Test and Pooled Variance

• Structural configuration:

  • Employs two completely separate, independent groups of subjects (e.g., experimental group versus control group).

• Formula for the independent samples $t$ test:   $t = \frac{(M_1 - M_2) - (\mu_{M_1} - \mu_{M_2})}{s_{(M_1 - M_2)}}$

  • $(M_1 - M_2)$ is the difference between sample means.

  • $(\mu_{M_1} - \mu_{M_2})$ is the hypothesized difference between population means under the null hypothesis, which is always $0$:     $\mu_{M_1} - \mu_{M_2} = 0$

• Formula for estimated standard error of the mean difference:   $s_{(M_1 - M_2)} = \sqrt{\frac{s_p^2}{n_1} + \frac{s_p^2}{n_2}}$

• Concept and formula for pooled variance ($s_p^2$):

  • Conceptual definition: An averaged variance weighted by degrees of freedom, granting the larger sample more influence ("say") in estimating the population error variance.

  • Formula:     $s_p^2 = \frac{SS_1 + SS_2}{(n_1 - 1) + (n_2 - 1)}$

  • Sum of squares ($SS$) computational equations:     $SS_1 = \sum X_1^2 - \frac{(\sum X_1)^2}{n_1}$     $SS_2 = \sum X_2^2 - \frac{(\sum X_2)^2}{n_2}$

• Degrees of freedom calculation:   $df = (n_1 - 1) + (n_2 - 1) = n_1 + n_2 - 2$

• Step-by-step calculation workflow:

  1. Calculate the mean for sample 1: $M_1 = \frac{\sum X_1}{n_1}$.

  2. Calculate the mean for sample 2: $M_2 = \frac{\sum X_2}{n_2}$.

  3. Calculate the sum of squares for sample 1 ($SS_1$).

  4. Calculate the sum of squares for sample 2 ($SS_2$).

  5. Calculate pooled variance ($s_p^2$) using pooled degrees of freedom.

  6. Compute the estimated standard error of the difference ($s_{(M_1 - M_2)}$).

  7. Calculate the $t$ ratio.

• Confidence interval formula for independent samples:   $(M_1 - M_2) \pm (t_{\text{conf}})(s_{(M_1 - M_2)})$

Independent Samples t Test Empirical Studies

• Seery, Holman, and Silver (2010) Life Adversity and Well-Being Study:

  • Theoretical rationale: Investigates the empirical validity of the phrase "That which doesn't kill us makes us stronger." Prior literature indicated individuals with a history of adversity exhibit superior mental health and well-being over individuals with negligible adversity history.

  • Group categorization (based on adverse events during the prior two years):

  • Low Adversity Group ($\le 2\text{ negative life events}$):

    • Sample values ($n_1 = 12$): $\{7, 8, 7, 2, 6, 9, 7, 10, 5, 0, 10, 8\}$

  • Moderate Adversity Group ($3\text{ to } 5\text{ negative life events}$):

    • Sample values ($n_2 = 11$): $\{4, 9, 12, 8, 9, 13, 12, 13, 13, 7, 6\}$

  • Analytical criteria: Evaluate differences in well-being at $\alpha = .05$ with pooled degrees of freedom:     $df = 12 + 11 - 2 = 21$

• Social Conformity and Authority Compliance in a Committee Atmosphere:

  • Experimental arrangement: Adapting classic obedience protocols where participants deliver escalating shocks ranging from 0 to 25 (0 indicates immediate refusal; 25 denotes full compliance to the investigator in a lab coat).

  • Two between-subjects conditions:

  • Committee Condition: Shocks are administered only after an affirmative group decision involving one naive subject and two experimental confederates who uniformly endorse the naive participant's choice.

  • Solitary Condition (Control): Shocks are administered following the decision of a single, uninfluenced naive subject.

  • Compliance scores ($N = 12$ total participants randomly assigned equally):

  • Committee ($n_1 = 6$): $\{2, 5, 20, 15, 4, 10\}$

  • Solitary ($n_2 = 6$): $\{3, 8, 7, 10, 14, 0\}$

  • Analytical criteria: Test the null hypothesis at $\alpha = .01$ with pooled degrees of freedom:     $df = 6 + 6 - 2 = 10$

Decision Framework for Selecting the Appropriate Statistical Test

• Decision criteria hierarchy:

  • Group count assessment:

  • If examining one group ($1\text{ group}$):

    • Is the population standard deviation ($\sigma$) known?

    • Yes: Execute a $Z$ test: $Z = \frac{M - \mu}{\sigma_M}$.

    • No: Execute a Single Sample $t$ test: $t = \frac{M - \mu}{s_M}$.

  • If examining two groups ($2\text{ groups}$):

    • Are scores matched, paired, or repeated across subjects before and after an intervention?

    • Yes: Execute a Dependent Samples $t$ test: $t = \frac{M_D - \mu_{M_D}}{s_{M_D}}$.

    • No: Execute an Independent Samples $t$ test: $t = \frac{(M_1 - M_2) - (\mu_1 - \mu_2)}{s_{(M_1 - M_2)}}$.

  • If examining three or more groups ($\ge 3\text{ groups}$):

    • Execute an Analysis of Variance (ANOVA).