T-Tests Notes

Scales of Measurement and Statistical Tools

• The scale of measurement of a dependent variable (Nominal, Ordinal, Interval, and Ratio) is a clue for selecting the appropriate statistical tool.

Types of t-tests

• One Sample t-test: Determines if a sample mean differs significantly from a hypothesized value.

• Independent Samples t-test: Determines if the means of two independent groups differ significantly.

• Paired Samples t-test: Determines if the mean of one group differs across repeated tests.

One Sample t-test

• Hypotheses:

  • Null Hypothesis ($H_0$): $\mu = 0$, where $\mu$ is the population mean.

  • Alternative Hypothesis ($H_1$): $\mu \neq 0$

Null Probability Distribution

• Variable Type: Interval/Ratio Dependent Variable (x)

• $\bar{X}$ = sample mean

• $\mu$ = population mean under the null hypothesis

• $\hat{\sigma}$ = sample standard deviation

• N = sample size

• Degrees of Freedom (df): $df = N - 1$ (Think independence)

Applying the One Sample t-test

• Example 1:

  • $H_0: \mu = 0$

  • $H_1: \mu \neq 0$

  • $t = (0.452 - 0) / 0.329$

  • $t(7) = 1.374$, $p = 0.212$

• Example 2:

  • $H_0: \mu = 4$

  • $H_1: \mu \neq 4$

  • $t = (4.401 - 4) / 0.208$

  • $t(7) = 1.927$, $p = 0.095$

• Critical values at t(7) = +/- 2.365 for $\alpha = 0.05$

Effect Size: Cohen's d

• Cohen's d is reported to estimate effect size.

• It distills a raw mean difference into standard deviation (SD) units.

• Formula: $d = (\bar{X} - \mu) / s$, where s is the sample standard deviation.

• Example:

  • $d_{wmc} = (4.401 - 4) / 0.588 = 0.681$

  • $d_{Precision} = (0.452 - 0) / 0.930 = 0.486$

Confidence Interval

• Confidence Intervals (e.g., 95% CI) combine sample data and the critical test statistic for $\alpha = 0.05$.

• If the same study were run 100 times, the true population mean would be within the computed interval 95 times.

• Example:

  • $H_0: \mu = 4$ for working memory capacity.

  • t(7) critical value = +/- 2.365

  • 95% CI = $4.401 \pm 2.365 * (0.208) = [3.89, 4.91]$

Assumptions

• Normality: The population distribution is assumed to be normal (bell-shaped and centered at the null hypothesized value).

  • This assumption can be tested.

  • If violated, a non-parametric version of the t-test can be used.

  • The t-test is robust to violations of this assumption.

• Independence: Observations are independently sampled.

  • This assumption cannot be formally tested.

  • If violated, the outcome of the t-test is invalid.

Independent Samples t-test

Null Probability Distribution

• Variable Type: Interval/Ratio Dependent Variables ($X<em>1$ & $X</em>2$)

• $\bar{X_1}$ = sample mean for group 1

• $\bar{X_2}$ = sample mean for group 2

• SE = pooled standard error

• $n_1$ = sample size group 1

• $n_2$ = sample size group 2

• Degrees of Freedom (df): $df = (n<em>1 - 1) + (n</em>2 - 1)$ (Think independence)

Assumptions

• Normality: The population distribution is assumed to be normal (i.e., bell shaped and centered at the null hypothesized value).

  • We can test this assumption.

  • If this assumption is violated we can use a non-parametric version of the t-test.

  • The t-test is robust to violations of this assumption

• Independence: Observations are independently sampled.

  • This assumption cannot be formally tested.

  • If this assumption is violated then the outcome of the t-test is invalid.

• Homogeneity of Variance: The population standard deviation is equal between the two groups.

  • This assumption can be tested.

  • If this assumption is violated, there are solutions (parametric and non-parametric).

Paired Samples t-test

• Hypotheses:

  • Null Hypothesis ($H<em>0$): $\mu</em>D = 0$, where $\mu_D$ is the population mean of difference scores.

  • Alternative Hypothesis ($H<em>1$): $\mu</em>D \neq 0$

Null Probability Distribution

• Variable Type: Interval/Ratio Dependent Variable (Difference scores)

• $\bar{D}$ = sample mean of difference scores

• $se(\bar{D})$ = standard error of difference scores

• N = sample size of difference scores

• Degrees of Freedom (df): $df = N - 1$ (Think independence)

Assumptions

• Normality: The population distribution of the difference scores is normal (bell-shaped and centered at the null hypothesized value).

  • Can test this assumption.

  • If violated, use a non-parametric alternative.

  • T-test is robust to violations of this assumption.

• Independence: Measurable observations are independently sampled.

  • Cannot formally test this assumption.

  • If violated, the outcome is invalid.

• No Significant Outliers in the Difference Scores: Outliers can distort results, especially in small samples.

  • Test by looking at a frequency distribution of the difference scores.

  • Test formally by computing z-scores for the difference scores, ensuring all z-scores fall between +/- Z = 2. (i.e., -2 < z < 2)