Independent Sample T-Tests

INDEPENDENT SAMPLE T-TESTS

Introduction

  • Course: PSY207 - Psychological Statistics

  • Instructor: Tim Pruitt, Ph.D.

Types of Samples for Mean Comparisons

Independent Samples
  • Also known as: Between-subjects or Between-groups.

  • Characteristics:
      - Observations are mutually exclusive with no overlap or connection.

Dependent Samples
  • Also known as: Within-subjects or Within-groups.

  • Characteristics:
      - Observations have some overlap or connection to each other.
      - Examples include repeated measures and matched groups.

Key Features
  • Separate groups of participants are used for different treatment conditions.

  • Random assignment is employed to place participants in treatment conditions.

  • Only one score is obtained for each participant.

Data Requirements for Independent Sample t-Tests

  • Data should be of interval or ratio scale.

  • Observations (scores) within each sample must be independent:
      - Since the samples consist of different participants, they maintain independence.

Assumptions of Independent t-Test

Core Assumptions
  • Random Sampling: The sample must represent the population.

  • Normality: The data should be approximately normally distributed (this is more flexible with larger samples).

  • Equal Variances: The populations must have equal variances, known as homogeneity of variance.
      - If the variances are significantly different, they are termed heterogeneous.

Sample Variability Measures

Calculation of Variability
  • Sample Variance (

s2=racSSn−1s^2 = rac{SS}{n - 1}

): Calculated using the sums of squares (SS).

  • Sample Standard Deviation (

s=extsqrt(s2)s = ext{sqrt}(s^2)

): The square root of the sample variance.

Pooled Variance

Definition & Requirement
  • Pooled Variance combines variance estimates from each sample. It requires that:
      - Variance from each sample is approximately equal.
        - A rule of thumb is: one variance should not be more than twice the other.

Calculation Formulas
  • Pooled Variance (

sp2s_p^2

) is calculated as:
  - sp2=racs12(n1−1)+s22(n2−1)n1+n2−2s_p^2 = rac{s_1^2(n_1-1) + s_2^2(n_2-1)}{n_1 + n_2 - 2}

  • Pooled Standard Deviation (

sps_p

):
  - sp=extsqrt(sp2)s_p = ext{sqrt}(s_p^2)

Pooled Standard Error

Calculation
  • Pooled Standard Error (

sM1−M2s_{M_1-M_2}

) can be calculated using:
  - With pooled variance:
    - sM1−M2=spext(rac1n1+rac1n2ext)s_{M_1-M_2} = s_p ext{(} rac{1}{n_1} + rac{1}{n_2} ext{)}
  - With pooled standard deviation:
    - sM1−M2=spext(rac1n1+rac1n2ext)s_{M_1-M_2} = s_p ext{(} rac{1}{n_1} + rac{1}{n_2} ext{)}

Unpooled Standard Error

Usage
  • An unpooled standard error is used when sample variances are significantly dissimilar:
      - sM1−M2=extsqrt(racs12n1+racs22n2)s_{M_1-M_2} = ext{sqrt}( rac{s_1^2}{n_1} + rac{s_2^2}{n_2} )

  • Importance of the unpooled method includes preventing inaccuracies in estimates and minimizing Type I and Type II error rates.

Hypothesis Testing in t-Tests

Models for Null and Alternative Hypotheses
  • Null Hypothesis (H0): Assumes no difference in means (

extH0:extμ1−extμ2=0ext{H0: } ext{μ}_1 - ext{μ}_2 = 0

).

  • Alternative Hypothesis (H1): Assumes difference exists (

extH1:extμ1−extμ2<br>eq0ext{H1: } ext{μ}_1 - ext{μ}_2 <br>eq 0

).

Test Statistic Computation
  • The test statistic formula:

t=rac(Mˉ<em>1−Mˉ2)−(extμ1−extμ2)s</em>M1−M2t = rac{(\bar{M}<em>1 - \bar{M}_2) - ( ext{μ}_1 - ext{μ}_2)}{s</em>{M_1-M_2}}

Degrees of Freedom

Degrees of Freedom Calculation
  • Combined degrees of freedom from each sample:
      - df=(n1−1)+(n2−1)extORdf=(n1+n2)−2df = (n_1 - 1) + (n_2 - 1) ext{ OR } df = (n_1 + n_2) - 2

  • If variances are unequal, the calculation may require more advanced statistical software (e.g., SPSS, JASP).
      - A conservative measure might involve using:
        - dfunpooled=extmin(n1−1,n2−1)df_{unpooled} = ext{min}(n_1 - 1, n_2 - 1)

Effect Size Measures

Cohen’s d Definition
  • Cohen's d is calculated as:

d=rac∣Mˉ1−Mˉ2∣spext(usingpooledstandarddeviation)d = rac{|\bar{M}_1 - \bar{M}_2|}{s_p} ext{ (using pooled standard deviation)}

Variance Accounted For (r²)
  • Calculation:

r2=ract2t2+dfr^2 = rac{t^2}{t^2 + df}

Interpretation Ranges
  • Cohen's d:
      - d < |0.2|: Small   - |0.2| < d < |0.8|: Medium   - d > |0.8|: Large

  • Variance Accounted For (r²):
      - r² = 0.01: Small
      - r² = 0.09: Medium
      - r² = 0.25: Large

Confidence Intervals in t-Tests

Estimation of Differences
  • Confidence intervals estimate the difference between population means (

extμ1−extμ2ext{μ}_1 - ext{μ}_2

):
  - Formula:

CI=Mˉ<em>1−Mˉ2ext±(t</em>criticalimessM1−M2)CI = \bar{M}<em>1 - \bar{M}_2 ext{ ± } (t</em>{critical} imes s_{M_1-M_2})

Note on Presentation
  • Alternative intervals can be generated for each group, especially useful in graphical presentations.

  • Always use a two-tailed critical value when assessing significance.

Statistical Software Outputs

Interpretation
  • Outcomes from statistical software might present findings such as non-significant differences between treatment and control groups (e.g., p > .05).

  • Noteworthy tests include:
      - Levene’s Test: Used to test homogeneity of variances. If significant (p < .05), use an alternative (Welch's t-test).
      - Shapiro-Wilk Test: Determines normality of each group.

Comedian Example Study

Context and Data
  • Study on the likability of comedians based on content type:
      - Neutral Comedian:
        - Mean Rating (M): 5.2,
        - Variance (s²): 2.17,
        - Sample Size (n): 15.
      - Offensive Comedian:
        - Mean Rating (M): 2.75,
        - Variance (s²): 2.21,
        - Sample Size (n): 12.

Hypothesis Statements
  1. Null Hypothesis (H0): No difference between offensive and neutral comedians.

  2. Alternative Hypothesis (H1): A difference exists between offensive and neutral comedians.

Calculating Degrees of Freedom
  • df=(15−1)+(12−1)=14+11=25df = (15 - 1) + (12 - 1) = 14 + 11 = 25

t Critical Value Determination
  • For a two-tailed test at α = .01,

  • The critical t value: tcritical=±2.787t_{critical} = ±2.787

Test Statistic Calculation
  1. Calculate Estimated Standard Error:
       Using pooled variances, where:
       - sp2=rac(2.17imes14)+(2.21imes11)25s_{p}^2 = rac{(2.17 imes 14) + (2.21 imes 11)}{25}

  2. Conduct t-test:
       - t=rac(2.75−5.2)0.57=−4.30t = rac{(2.75 - 5.2)}{0.57} = -4.30

Decision Rule
  • Since the calculated t value (-4.30) is less than the critical t value (-2.787), reject H0.

Effect Size Calculation
  • Calculate Cohen’s d:

d=rac∣(2.75−5.2)∣sp2=∣rac−2.451.48∣=1.67ext(LargeEffect)d = rac{|(2.75 - 5.2)|}{s_p^2} = | rac{-2.45}{1.48}| = 1.67 ext{ (Large Effect)}

Confidence Interval at 99%
  • Lower CI:

CILower=Mˉ<em>1−Mˉ2−(t</em>criticalimessM1−M2)CI_{Lower} = \bar{M}<em>1 - \bar{M}_2 - (t</em>{critical} imes s_{M_1-M_2})

  • Upper CI:

CIUpper=Mˉ<em>1−Mˉ2+(t</em>criticalimessM1−M2)CI_{Upper} = \bar{M}<em>1 - \bar{M}_2 + (t</em>{critical} imes s_{M_1-M_2})

Overall Interpretation
  • The results indicate a significant difference in likability between comedians based on their content, with a large effect size reported.

Caffeine Reaction Time Example
  • Hypothesis: Caffeine increases reaction speed.

  • Group Data:
      - Caffeinated Group:
        - Mean (M): 681.2 ms, Standard Deviation (s): 35.44, Sample Size (n): 8.
      - Control Group:
        - Mean (M): 710.5 ms, Standard Deviation (s): 51.26, Sample Size (n): 9.

  • If significant at α = .05, effect size and CI r² and 95% CI should be calculated accordingly.