Psychological Research Methods and Analysis - Paired-Samples t-test & Type 1 and Type 2 Errors

Paired-Samples t-test, Type 1 and 2 Errors

One-sample t-test: Used to determine whether the mean calculated from sample data collected from a single group is different from a designated value (e.g., population parameter).
Independent-Samples t-test: Compares two groups of unpaired data (between-subjects design).
Paired-samples t-test: Compares two groups of paired data (within-subjects or repeated measures design).

Compares means of two measures taken from the same individual/related units.
Determines if the difference between paired means is statistically significant.
The score in one group is tied to a particular score in the other group; knowledge of one score helps to predict the other.
Used in within-subjects/repeated measures designs.

Principles of NHST apply, including a null hypothesis (H0) and an alternative hypothesis (H1).
H0: µ1 = µ2 (the paired population means are equal).
H1: µ1 ≠ µ2 (the paired population means are not equal).

For within-subjects designs:
- Statistical difference between two time points (e.g., pre- and post-intervention).
- Statistical difference between two conditions.
- Statistical difference between a matched pair.
More power than independent samples t-tests because it eliminates variation between samples.

Investigating the effect of background music (IV) on time to complete a maths test (DV).
The same participants complete the test in both a silent condition and a music condition.
Looking at whether the difference between means is significantly different from chance.

Mean Difference: The difference between the group means.
t: Test statistic, calculated as t = (mean \, difference / SE \, of \, mean), indicates how many standard errors the mean difference is away from zero.
df: Degrees of freedom, calculated as df = N - 1.
Sig. (2-tailed): The p-value, representing the probability of observing the difference if H0 is true.
Decision rule: if p-value is less than alpha level, it is a statistically significant result.

Example: t (11) = 3.07, p = .011
Including effect size (Cohen's d) and confidence intervals provides a more complete picture.
Example: t (11) = 3.07, p = .011, d = -0.43, 95% CI [-0.79, -0.06]

Statistically significant result: 95% CI does not contain zero.
Statistically non-significant result: 95% CI contains zero.
If 0 (no effect) is a plausible estimation in the population, results will not be significant.

Whenever the 95% CI does not contain zero, the t-test will be statistically significant (at the .05 α level).
Whenever the 95% CI contains zero, the t-test will be statistically non-significant (at the .05 α level).

Type I Error (False Positive): Rejecting H0 when H0 is actually true.
Type II Error (False Negative): Failing to reject H0 when H1 is actually true.

What is the true effect in the population (should there be a significant difference between groups or not)?
What would the study conclude based on p-value?
How does this fit with the true effect in the population?
If an error was made, was this a Type I (false positive) or Type II (false negative)?

Study: IV - Weather forecast for tomorrow in Sunderland (sunny or cloudy), DV - Reaction time of students; p = .03 (α =.05) - Type I Error.
Study: IV - Basketball players and children aged 5, DV - Height; p = .08 (α =.05) - Type II Error.

Probability of a Type I error is equal to the significance level α.
Probability of a Type I error is independent of the sample size N.
When using a 5% significance level, there is a 5% chance of rejecting the null hypothesis if the null hypothesis is true.
Increasing α increases the Type I error rate.