T-Tests for Independent Means: Estimating Population Variance, Effect Size and Power

Estimating Population Variance

• When conducting a t-test for independent means, we are dealing with two populations.

• Independent t-tests are frequently used in research.

• Calculations are more complicated due to having two sets of means, variances, standard deviations, and degrees of freedom.

Pooled Estimate of Population Variance

• The most crucial aspect is the variances.

• We calculate a pooled estimate of population variance.

• Assumption: The two populations have the same variance. This is critical. If the populations do not have the same variances, an additional step is required, increasing the calculation complexity. However, in this course, we assume equal variances.

• Statistical software (SPSS, Stata, R) can handle cases with unequal variances.

Steps to Find Standard Error (SE) Difference

• This involves several formulas.

  1. Calculate two unbiased estimated population variances (familiar from t-tests with a single mean). This involves calculating $s<em>1^2$ and $s</em>2^2$, the unbiased estimators of the population variances for each group.

  2. Calculate degrees of freedom (DF) total.

     • $DF_{total}$ formula is different from single mean t-tests.

     • $DF<em>1 = n</em>1 - 1$ (where $n_1$ is the sample size of group 1).

     • $DF<em>2 = n</em>2 - 1$ (where $n_2$ is the sample size of group 2).

  3. Calculate the pooled variance $s_p^2$. Also known as "s squared pooled formula",

     • $s<em>p^2 = \frac{DF</em>1 * s<em>1^2 + DF</em>2 * s<em>2^2}{DF</em>{total}}$.

  4. Calculate the standard deviation of the distribution of differences between means ($SE_{difference}$).

     • $SE<em>{difference} = \sqrt{\frac{s</em>p^2}{n<em>1} + \frac{s</em>p^2}{n_2}}$.

• Important Notes:

  • Step 2 (calculating pooled estimate of population variance): Uses degrees of freedom.

  • Step 3 (calculating $SE_{difference}$): Uses sample size.

  • Avoid confusing degrees of freedom and sample size.

Formula for Calculating t-score

• The t-score formula is relatively straightforward once $SE_{difference}$ is calculated.

• $t = \frac{M<em>1 - M</em>2}{SE<em>{difference}}$. Where $M</em>1$ and $M_2$ are the sample means.

• Numerator: Mean difference (mean of sample 1 minus mean of sample 2).

• Population mean difference is assumed to be zero, hence its omission from the formula.

• $DF<em>{total} = DF</em>1 + DF_2$.

Independent vs. Dependent Means T-tests

• Similarities: Both involve two sets of scores.

• Differences:

  • Independent means: Scores from entirely different populations.

  • Dependent means: Scores are related (e.g., same people, paired individuals).

• Order of Calculations:

  • Independent means: Calculate means of samples first, then $SE_{difference}$, then t-score.

  • Dependent means: Calculate difference scores first, then the mean of the difference scores, then follow the t-test for a single mean procedure.

Effect Size

• Interpretation of effect size remains the same:

  • Around 0.2: Small effect.

  • Around 0.5: Medium effect.

  • Around 0.8: Large effect.

• Formula:

  • $Cohen's\;d = \frac{M<em>1 - M</em>2}{\sqrt{s<em>p^2}}$, where $s</em>p^2$ is the pooled variance.

  • The numerator is the difference between two population means (estimated).

  • The denominator is the square root of the $s^2$ pooled (calculated previously).

Power

• Refer to Table 9.4 for power calculations.

• Assumptions:

  • Equal number of participants in each group (e.g., 10 kittens and 10 puppies).

  • Power is greatest when participants are divided into two equal groups.

• Unequal Sample Sizes:

  • Use harmonic means when the groups do not have the same sample size.

  • Formula:

    • $Harmonic\;Mean = \frac{2}{\frac{1}{n<em>1} + \frac{1}{n</em>2}}$.

    • $n_1$: Sample size from the first sample.

    • $n_2$: Sample size from the second sample.