Lecture 6 Notes

Comparing Means: T-Tests

• Focus on differentiating statistical tests.
• Last week: hypothesis testing, decision errors, statistical power, confidence intervals, and meta-analyses.
• Next three weeks: specific tests and their appropriate situations.
• Today: tests involving comparing means.
  • T-tests: comparing means of one or two samples to each other or to a population.
  • General Linear Model: testing differences between groups with more than two groups.

T-Tests

• General form of a t-statistic:
  • A ratio of the sampling error divided by the standard error of that statistic.
  • Sampling error: the difference between an observed sample statistic and the hypothesized population parameter.
  • Standard error: the standard deviation of the sampling distribution of the statistic.
  • Formula: $t = \frac{{\text{sample statistic} - \text{population parameter}}}{{\text{estimated standard error of the statistic}}}$
• T-tests are used in various scenarios, including linear regression to test the significance of intercepts and slopes.

Types of T-Tests

• One Sample T-Test:
  • Comparing the mean of one sample to the mean of a population. (e.g., ginseng and IQ)
• Dependent Measures (Paired Samples) T-Test:
  • A one-group design where two measures are taken from the same person and compared. (e.g., before and after, condition A vs. condition B)
• Independent Measures (Independent Samples) T-Test:
  • A two-group design where the means of two independent groups are compared in a between-subjects format.

One Sample T-Test

• Formula: $t = \frac{{\text{sample mean} - \text{population mean}}}{{\text{standard error of the mean}}}$
• Similar to a z-score but used when the population standard deviation is unknown and estimated from the sample.
• Requires considering degrees of freedom.

Z-Test vs. T-Test

• Z-Test: used when the population standard deviation is known.
• T-Test: used when the population standard deviation is not known and must be estimated. In this case, degrees of freedom are important.

Degrees of Freedom

• Refers to the pieces of independent information that go into a calculation.
• Estimates from larger samples are more stable than estimates from smaller samples.
• Degrees of freedom influences the shape of the distribution.
• For a one-sample t-test, degrees of freedom $df = n - 1$.
• Variance is equal to the sum of squared deviations over $n-1$.
• Degrees of freedom influences what the critical value is.

One Sample T-Test Example: UCSD Students Exercise

• Research Question: Do UCSD students exercise more than the average American adult?
• Sample: 57 students
• Sample Mean: 3.39 hours/week
• Sample Standard Deviation: 3.56
• Population Mean: 2 hours/week
• Null Hypothesis: UCSD students exercise the same amount as the average American adult.
• Standard Error: $SE = \frac{s}{\sqrt{n}} = \frac{3.56}{\sqrt{57}} = 0.47$
• T-Statistic: $t = \frac{\bar{x} - \mu}{SE} = \frac{3.39 - 2}{0.47} = 2.95$
• Degrees of Freedom: $df = n - 1 = 56$
• P-Value: 0.0046
• Interpretation: There is a 0.46% chance of getting a test statistic this extreme if UCSD students exercise 2 hours/week.
• Conclusion: Results are statistically significant (p < 0.05), suggesting UCSD students exercise more.
• Reporting: t(56) = 2.95, p = 0.0046

Dependent Samples (Paired Samples) T-Test

• Used in within-subjects designs: typically when you can compute a difference score.
• Example: Blood pressure drug effectiveness
  • Measure blood pressure after placebo and after the active drug.
  • Compute a difference score for each person.
  • Compare the mean difference score to what you would get under the null hypothesis.
• Important: you need to have two measures of the same variable from the same person.

Paired Samples T-Test Statistic

• Null Hypothesis: The drug has no effect; the mean difference score is zero.
• Alternative Hypothesis: The mean difference score is not zero.
• Formula: $t = \frac{{\bar{D} - \mu_D}}{{\text{standard error of difference}}}$, where $\bar{D}$ is the mean difference score.

Paired Samples T-Test Example: UCSD Students Cooking vs. Eating Out

• Research Question: Do UCSD students cook for themselves more often than they eat out?
• Sample: 57 students
• Mean Cooking: 7.65 times/week
• Mean Eating Out: 6.06 times/week
• Mean Difference: 1.59
• Standard Deviation of Difference Scores: (provided)
• Standard Error: $SE = \frac{{\text{standard deviation of difference scores}}}{{\sqrt{n}}}$
• T-Statistic: $t = \frac{{1.59 - 0}}{{\text{SE}}}$ (assuming the standard error calculation is done correctly)
• P-Value: 0.12
• Interpretation: There is a 12% chance of observing this difference if the means are equal.
• Conclusion: Results are not statistically significant (p > 0.05).
• Reporting: t(56) = 1.56, p = 0.12
• We do not accept the null hypothesis as true, only fail to reject it.

Independent Samples T-Test

• Comparing two different kinds of treatments or two different groups of people.
• Example: Comparing drug A and drug B.
• Goal: Determine if the difference between the two means is greater than what would be expected by chance under the null hypothesis.

Independent Samples T-Test Hypotheses

• Null Hypothesis: There is no difference between the groups; the difference between means is zero.
• Alternative Hypothesis: The difference between means is not zero.

Independent Samples T-Test Statistic

• Sample Statistic: The difference between sample means.
• Population Parameter: The hypothesized difference between population means (zero under the null hypothesis).
• Formula: $t = \frac{{\text{difference between sample means}}}{{\text{estimated standard error of the difference}}}$
• The denominator involves two different standard deviation estimates.
• Degrees of freedom calculation is affected.

Independent Samples T-Test Example: On-Campus vs. Off-Campus Work Hours

• Research Question: Do students who live off-campus work more hours per week than students who live on-campus?
• Sample: 57 students
• Off-Campus Mean: 5.03 hours/week
• On-Campus Mean: 1.22 hours/week
• Difference of Means: 3.81 hours
• Pooled Standard Error: 1.45 (provided)
• T-Statistic: $t = \frac{{3.81}}{{1.45}} = 2.62$
• Degrees of Freedom: 55 (n - 2 because two means were calculated)
• P-Value: 0.011
• Conclusion: Statistically significant; evidence that off-campus residents work more hours.
• Reporting: t(55) = 2.62, p = 0.011
• Each person can only be in one of the groups.

Comparison of T-Tests

• One Sample: Comparing a sample mean to a hypothesized population mean.
• Dependent Measures: Looking at a mean difference score.
• Independent Samples: Looking at a difference in two means.
• All are variations of the same general form: An observed difference over a standard error of the difference.

Peer Instruction: Choosing the Correct T-Test

• Scenario 1: Alcohol and Learning Ability
  • Independent Samples T-Test: Because participants are randomly assigned to either an alcohol group or a control group, not both.
• Scenario 2: Blood Oxygen and Lactic Acid
  • None of the Above: Because blood oxygen and lactic acid are different variables, a t-test is not appropriate to analyze their relationship. A correlation test would be more appropriate.
• Scenario 3: Brussels Sprouts
  • Paired Samples T-Test: Attitudes are measured before and after an intervention on the same individuals for the same measure.

General Linear Model (GLM) and ANOVA

• Compares the means of three or more things at the same time.

General Linear Model

• A way of representing patterns in data.
• Variation in scores partitioned into chunks explained by a model and unexplained error.
• Model: A prediction (simple or complex).
• Mean of a distribution as a simple model.
• $Y<em>{ij} = \mu + \tau</em>j + \epsilon_{ij}$
  • $Y_{ij}$ = Score on y for the $i^{th}$ person in the $j^{th}$ group
  • $\mu$ = Grand mean in the population.
  • $\tau$ = Treatment offset.
  • $\epsilon_{ij}$ = Residual error.
• Sample Estimation:
  • $Y<em>{ij} = \bar{Y} + T</em>j + E_{ij}$
  • $\bar{Y}$ = the grand mean in the data
  • $T_j$ = treatment offset for some group
  • $E_{ij}$ = estimated residual error

Components of GLM

• Grand Mean: The average of all data points.
• Treatment Offset: How the mean of a group is different from the overall mean.
• Residual Error: How an individual's score differs from the mean of their group.

Toy Example: Calculating GLM Components

• Data: 9 subjects in 3 groups (A, B, C) with scores of 3-5, 7-9, and 11-13 respectively.
• Group Means: A=4, B=8, C=12
• Grand Mean: 8
• Treatment Offsets: A=-4, B=0, C=4
• Residual Errors: Individual deviations from group means.
• Predicted Score: $\text{Grand Mean} + \text{Treatment Offset} = \text{Group Mean}$
• Residual Error: How far off the guess is.

Peer Instruction - GLM Definitions

• Treatment Offset vs. Group Mean:
  • Group mean is the combination of the grand mean and the treatment offset.
• Predicted Score vs. Group Mean:
  • Group mean is a predicted score.

F Ratio

• In a hypothesis test, the null hypothesis is that the groups are all the same (treatment offsets are zero).
• Alternative Hypothesis: That they are not all zero.
• Goal is to determine the likelihood that the null hypothesis is true.

Peer Instruction - Implications of Rejecting the Null Hypothesis

• If we get p < α and we reject the null, the most we can say is "at least one of the groups has a different mean from the others."

ANOVA (Analysis of Variance)

• Used to analyze differences between three or more groups at the same time.
• Compares variation between groups relative to variation within groups.
• The F ratio is variability between relative to variability within for each group.
• High within group variability: the group is wider
• Lower within the group variability: groups are narrower

Total Deviation

• Total deviation equals error deviation plus treatment deviation.
• Our treatment deviation is equal to how different is the group mea from the grand mean.
• Error deviation: How does your observed score differ from the group mean.
• Total deviation: How does that person differ from the grand mean.
• Error deviation: Yi - ŷ
• Sums of Squares: $\text{Total SS} = \text{Treatment SS} + \text{Error SS}$
• The variance of the group offsets divided by the variance of the residuals.

F Ratio Logic

• $F = \frac{{\text{Mean Squared Treatment}}}{{\text{Mean Squared Error}}}$
• If groups are really different from each other, this ratio gets big. If groups are really similar to each other, this ratio gets small.

Reporting F Test Results

• General Form: $\text{F(df numerator, df denominator) = F-ratio, p = p-value}$
• Example: F(2, 6) = 48, p = 0.001
• Conclusion: We find evidence that at least one of the groups is different from the others.
• Follow up a significant ANOVA with pairwise comparisons (e.g., Tukey HSD) to determine which pairs of conditions differ significantly.