Independent-Measures T-Test Statistics and Procedures

Introduction to the Independent-Measures Design

When psychological research involves scores that are numerical values, the central research question typically concerns the mean difference between two sets of data. These sets of data are obtained through research designs categorized into two general types. First, the data sets may come from two completely separate groups of participants. Examples provided include comparing a sample of men with a sample of women, or comparing grades between a group of freshmen provided with laptop computers and a second group who were not provided with computers. Second, data can come from the same group of participants measured twice, such as measuring depression in a group of patients before therapy and again after 6 weeks of therapy. A design that utilizes a separate group of participants for each treatment condition or population is formally known as an independent-measures research design or a between-subjects design.

Questions and Discussion

These check-your-learning questions cover the fundamental concepts of independent-measures designs:

Which of the following describes an independent-measures research study? Answer: Two separate samples are used to obtain two groups of scores to represent the two populations or two treatment conditions being compared.
Which of the following is an example of an independent-measures design? Answer: a. Comparing performance scores for 6-year-old boys and 6-year-old girls. (Note: Options b and d describe repeated-measures, while c describes a single-sample scenario).
A repeated-measures study comparing two treatment conditions uses ___ group(s) of participants and obtains ___ score(s) for each participant. Answer: b. 1 group, 2 scores per participant.
Which of the following is the correct null hypothesis for an independent-measures t test? Answer: b. $\mu_1 - \mu_2 = 0$
One sample from an independent-measures study has $n = 9$ with a variance of $s^2 = 35$ . The other sample has $n = 3$ and $s^2 = 40$ . What is the df value for the t statistic? Answer: d. 10. (Calculation: $(n_1 - 1) + (n_2 - 1) = (9 - 1) + (3 - 1) = 8 + 2 = 10$ ).
Experimental scenario: Research by Zhong, Bohns, & Gino (2010) suggests people exhibit more dishonest behaviors in darkness. Participants were given 20 puzzles and paid $\$0.50$ for each solved in 5 minutes. Since reporting was self-monitored, it provided an opportunity to cheat. One group was in a well-lit room and another in a dimmed room. This scenario serves as the basis for analyzing independent-measures data.

The Null Hypothesis and the Independent Measures T Statistics

The goal of this design is to evaluate the mean difference between two populations or treatment conditions, denoted as $\mu_1$ and $\mu_2$ . The difference is expressed as $\mu_1 - \mu_2$ . The null hypothesis ( $H_0$ ) assumes no effect or difference, stated as $H_0 : \mu_1 - \mu_2 = 0$ . Although this is equivalent to $\mu_1 = \mu_2$ , the version equaling zero is preferred because it provides a specific numerical value for the t-statistic calculation. The alternative hypothesis ( $H_1$ ) states that a mean difference exists: $H_1 : \mu_1 - \mu_2 \neq 0$ or $\mu_1 \neq \mu_2$ .

Formulas for an Independent-Measures Hypothesis Test

The independent-measures t-statistic follows the same basic structure as the single-sample t-statistic, but it doubles the elements to accommodate two samples. While a single-sample t-test uses one sample mean ( $M$ ) to test against a population mean ( $\mu$ ), the independent-measures t-test uses the difference between two sample means ( $M_1 - M_2$ ) to evaluate the difference between two population means ( $\mu_1 - \mu_2$ ).

The formula for the independent-measures t-statistic is: $t = \frac{(M_1 - M_2) - (\mu_1 - \mu_2)}{s_{(M_1 - M_2)}}$

In hypothesis testing, the null hypothesis typically sets the population mean difference ( $\mu_1 - \mu_2$ ) to zero, allowing the formula to be simplified to: $t = \frac{M_1 - M_2}{s_{(M_1 - M_2)}}$

Calculating The Estimated Standard Error

The estimated standard error ( $s_{(M_1 - M_2)}$ ) in the denominator measures how accurately the sample mean difference represents the population mean difference. There are two sources of error because $M_1$ approximates $\mu_1$ and $M_2$ approximates $\mu_2$ . The error for each mean is measured by the estimated standard error of the mean ( $s_M$ ), calculated as: $s_M = \sqrt{\frac{s^2}{n}}$

For independent-measures, if sample sizes are exactly the same ( $n_1 = n_2$ ), the standard error is the square root of the sum of the individual errors: $s_{(M_1 - M_2)} = \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}$

However, this formula is biased if sample sizes are unequal because it treats both variances equally. According to the Law of Large Numbers, statistics from larger samples are more accurate estimates of population parameters. Therefore, the variance from a larger sample should carry more weight.

Pooled Variance and Degrees of Freedom

To correct for bias in unequal sample sizes, sample variances are combined into a "pooled variance" ( $s_p^2$ ). This is an average of the two variances weighted by their respective degrees of freedom ( $df$ ). The formula for pooled variance is: $s_p^2 = \frac{SS_1 + SS_2}{df_1 + df_2}$

Where $SS$ is the sum of squared deviations. Using pooled variance, the unbiased estimated standard error is: $s_{(M_1 - M_2)} = \sqrt{\frac{s_p^2}{n_1} + \frac{s_p^2}{n_2}}$

Example of Pooled Variance (Equal Sizes): Sample 1: $n = 6, SS = 50, s^2 = 10$ . Sample 2: $n = 6, SS = 30, s^2 = 6$ . $s_p^2 = \frac{50 + 30}{5 + 5} = \frac{80}{10} = 8$ .

Example of Pooled Variance (Unequal Sizes): Sample 1: $n = 3, SS = 20, s^2 = 10$ . Sample 2: $n = 9, SS = 48, s^2 = 6$ . $s_p^2 = \frac{20 + 48}{2 + 8} = \frac{68}{10} = 6.8$ .

The degrees of freedom for the independent-measures t-statistic are finding by adding the df for both samples: $df = df_1 + df_2 = (n_1 - 1) + (n_2 - 1) = n_1 + n_2 - 2$

Summary Comparison: Single-Sample vs. Independent-Measures

For a single-sample t-test, the sample data is $M$ , the hypothesized parameter is $\mu$ , and error is $\sqrt{\frac{SS}{df \times n}}$ . The variance is $s^2 = \frac{SS}{df}$ and the degrees of freedom is $df = n - 1$ .

For an independent-measures t-test, the sample data is $(M_1 - M_2)$ , the hypothesized parameter is $(\mu_1 - \mu_2)$ , and the error is $\sqrt{\frac{s_p^2}{n_1} + \frac{s_p^2}{n_2}}$ . The pooled variance is $s_p^2 = \frac{SS_1 + SS_2}{df_1 + df_2}$ and the degrees of freedom is $df = df_1 + df_2$ .

Effect Size and Confidence Intervals

Effect size is measured using Cohen’s d or the percentage of variance ( $r^2$ ). Cohen’s d estimates the magnitude of the difference between group means relative to the standard deviation. The formula uses the pooled standard deviation (the square root of pooled variance): $\text{Estimated Cohen's d} = \frac{M_1 - M_2}{\sqrt{s_p^2}}$

This is used when comparing means to determine the magnitude of difference, such as comparing test scores from two teaching methods. Percentage of variance ( $r^2$ ) measures how much of the variability in scores can be explained by treatment effects, typically used in analyzing the relationship between independent and dependent variables.

Comprehensive Step-by-Step Calculation Example

Study: Comparing people who have experienced counseling (Group X1) vs. those who have not (Group X2).

Data X1: $7, 8, 10, 8, 7, 6, 4, 7, 8, 9$ ( $n_1 = 10$ ) Data X2: $5, 4, 7, 6, 7, 5, 3, 4, 4, 5$ ( $n_2 = 10$ )

STEP 1: Compute the means. Total X1 = 74; $M_1 = \frac{74}{10} = 7.4$ Total X2 = 50; $M_2 = \frac{50}{10} = 5$

STEP 2: Compute the variance for each group. $\sum X_1^2 = 574, \sum X_2^2 = 266$ $s_1^2 = \frac{572}{10} - (7.4)^2 = 57.2 - 54.76 = 2.44$ $s_2^2 = \frac{266}{10} - (5)^2 = 26.6 - 25 = 1.6$ (Note: The transcript uses $572$ for the numerator in the $s_1^2$ calculation despite listing the sum as $574$ in the table).

STEP 3: Compute the standard error of difference between means ( $s_{M_1 - M_2}$ ). $s_{M_1 - M_2} = \sqrt{\frac{N_1 s_1^2 + N_2 s_2^2}{N_1 + N_2 - 2} \times \frac{N_1 + N_2}{N_1 N_2}}$ $s_{M_1 - M_2} = \sqrt{\frac{10(2.44) + 10(1.6)}{10 + 10 - 2} \times \frac{10 + 10}{10 \times 10}}$ $s_{M_1 - M_2} = \sqrt{\frac{24.4 + 16}{18} \times \frac{20}{100}} = \sqrt{\frac{40.4}{18} \times 0.2}$ $s_{M_1 - M_2} = \sqrt{2.2444 \times 0.2} = \sqrt{0.4488}$ $s_{M_1 - M_2} = 0.66999$

STEP 4: Compute the t-statistic. $t = \frac{M_1 - M_2}{s_{M_1 - M_2}}$ $t = \frac{7.4 - 5}{0.66999} = \frac{2.4}{0.66999} = 3.58$

STEP 5: Identify the critical value. $df = 10 + 10 - 2 = 18$ Alpha value ( $\alpha$ ) = $0.05$ . Consulting the t-table for $df = 18$ and two-tails $\alpha = 0.05$ : Critical value = $\pm 2.101$ .

STEP 6: Compare the t-statistic with the critical value and conclude. The computed $t = 3.58$ is greater than the critical value of $2.101$ . Since t > 2.101, the null hypothesis is rejected. Conclusion: There is a significant difference between those who have experienced counseling and those who have not.