Lecture 4: Testing Hypotheses: Independent Sample t-test

The Big Picture

• Weeks 1-5 focused on describing data:

  • Types of data variables.

  • Distribution (how data is spread out).

  • Normal Distribution (a common bell-shaped distribution).

  • Histogram (a bar graph showing data distribution).

  • Mean (average).

  • Standard Deviation (how spread out numbers are).

  • Skewness (measure of asymmetry of a distribution).

  • Samples & Populations (small group vs. entire group).

  • Sampling distribution of the mean (distribution of sample means).

  • Standard error of the mean (how much sample means vary).

  • 95% confidence interval (range likely to contain the true population mean).

• Weeks 3 & 4 covered testing hypotheses:

  • Hypothesis (an educated guess).

  • Null hypothesis (a statement of no effect).

  • Test statistics (a number summarizing the evidence).

  • One-sample T-Test (comparing a sample mean to a known value).

  • Independent samples T-Test (comparing means of two independent groups).

  • Paired samples T-Test (comparing means of two related groups).

  • Within and Between subject designs (how participants are assigned to groups).

  • One/Two-tailed tests (direction of the hypothesis).

  • Practical application assessed in coursework.

  • Knowledge assessed in MCQ.

• Weeks 3, 4, & 5 focused on interpreting and reporting:

  • Levene’s Test (tests if groups have equal variances).

  • Shapiro-Wilks Test (tests if data is normally distributed).

  • Degrees of freedom (number of independent pieces of information).

  • Assigning significance – p-values (probability of observing the results if the null hypothesis is true).

  • Effect sizes (how big the effect is).

  • Reporting T-Tests

Week-by-Week Topics

Outline & Objectives

1. Two sample hypotheses & study design

   • Comparing the means (averages) of two different groups

   • Using data to answer questions

2. Independent Samples T-Tests

   • How does a two-sample t-test work?

   • What are its assumptions?

   • How can we run this in Jamovi?

3. Paired Samples T-Test

   • A test for within subjects designs

   • (actually, a one-sample t-test in disguise)

Two Sample Hypotheses

• Comparing two groups.

Study Design

• Many tests assume that data observations are independent, meaning each data sample is a separate observation.

• If observations are related, for example because one participant did the experiment twice, this changes how we can interpret the results.

• Both approaches (independent and related observations) are valid, but require different tests.

Within-Subjects Design

• Everyone contributes to both conditions – repeated measures.

• Condition 1 and Condition 2.

Between-Subjects Design

• Each person contributes to a single condition.

• Condition 1 and Condition 2.

Within and Between Subject Design

• Between subjects:

  • Two independent groups of data points.

  • Each participant is in a single group and contributes a single data point.

• Within subjects:

  • Two dependent groups of data points.

  • Each participant completes two conditions and contributes two data points.

  • Sometimes called ‘repeated measures’.

Hypotheses

• A two-sample hypothesis is a statement about the means of two different samples.

• Typically, we’re interested in learning whether the mean of the groups are different.

Two-Sample Hypotheses Examples

• Independent Samples:

  • Football players run 200m faster than rugby players.

  • Reoffending rates are lower for prisons that focus on rehabilitation rather than punishment.

• Dependent Samples:

  • Students' attention spans are longer on days with fewer teaching sessions.

  • A new therapy increases calorie intake in people with Anorexia Nervosa.

Two-Sample NULL Hypotheses

• Independent Samples:

  • Football players run 200m in the same time as rugby players.

  • Reoffending rates are the same for prisons that focus on rehabilitation rather than punishment.

• Dependent Samples:

  • Students' attention spans are the same on days with fewer teaching sessions.

  • A new therapy doesn’t change calorie intake in people with Anorexia Nervosa.

Hypotheses - Summary

• We need to think about our statistics from the start of the study design phase of a project.

• The design of the experiment impacts which statistics we can use. Whether or not we have independent data samples is a key example.

• The distinction must be clear when writing hypotheses.

Computing an Independent Samples T-Test

• Comparisons of two group means, or a single mean to a reference value.

• Data must be interval or ratio type.

• Assumptions must be met.

• Our data must have an interpretable mean and standard deviation.

When to Use a T-Test - Assumptions

Comparing Two Independent Groups

• Demonstrates two groups' distributions.

Independent Samples T-Test

• An independent samples t-test is the difference between the two means of two groups of data, all divided by the standard error of that difference.

• It is a ratio between the size of the difference and the precision to which it is estimated.

• t(df) = \frac{X1 - X2}{Sp\sqrt{\frac{2}{N}}}

  • Where:

    • X1 is the Mean of Group 1.

    • X2 is the Mean of Group 2.

    • Sp is the Pooled Standard error of the difference.

Independent Samples T-Test - Standard Error

• The standard error of the difference is computed using the pooled standard deviation of the two groups.

• The pooled standard deviation is a single standard deviation to represent the variability in both groups – assuming that both groups have the same variability.

Interpreting T-Values

Homogeneity of Variance

• Student’s t-test assumes Homogeneity of Variance, i.e., the distributions of the two groups have the same standard deviation.

• Is that always fair?

Levene’s Test for Homogeneity of Variance

• Levene’s test assesses the null hypothesis that different groups of samples are from populations with equal variances (spread).

• A significant value for Levene’s test indicates that the groups are likely to have different variances – suggesting that a pooled estimate of standard deviation is not appropriate.

Welch’s Test

• Welch’s test uses an UNPOOLED measure of standard deviation which is valid when the groups have different variance.

• The unpooled standard deviation is valid whether the groups have equal variances or not.

• Welch’s t-test formula (similar to the regular t-test but uses an unpooled standard error of the difference).

Example: Grey Matter Volume and Age

• Dataset of MRI scans from people aged between 18 and 88 years old (https://cam-can.mrc-cbu.cam.ac.uk/).

• Compute the percentage of each person's brain that is composed of grey matter, white matter, and CSF.

• How is that composition different between younger and older adults?

• Grey matter volume changes with age.

Analysis in Jamovi

• We need 2 columns to do our analysis:

  • A categorical Grouping Variable (variable that divides people into groups).

  • A continuous Outcome Variable (variable that measures the outcome).

• We are comparing Grey Matter Volume between the Old and Young groups specified in ‘AgeGroup’ in this dataset.

Results

• A t-value of 17 suggests that the difference is enormous compared to the precision to which we can estimate it from this data.

• Strong evidence for a real effect.

• The effect is massive.

Visualising Homogeneity of Variance with descriptive plots

• Always a good idea to check the data out yourself as well. Descriptive statistics and plots are a useful tool for this.

Reporting Welch’s Test

• “An independent samples t-test was used to compare the grey matter volumes between the young and old participant groups. Levene’s test on these variables suggested that the assumption of equal variance was violated; F(1, 569) = 8.14, p=0.004. Welch’s t-test showed that grey matter volume was higher in the young (M=43.2, SD=1.63) compared to the old (M=40.2, SD=2.05) groups; t(462) = 19.1, p<0.001.”

Paired Samples t-test

• A paired samples t-test follows the same principle as the independent samples test.

• Used when we’re comparing the means of two dependent distributions – that is, when the same participants have contributed to each condition.

• In these cases, the assumptions of ‘independent samples’ are violated in the standard t-test.

Paired Samples t-test - Calculation

• We simply take the difference between each pair of samples and compute a one-sample t-test between the paired difference and zero.

• Jamovi and R can do this for us by specifying that we’re running a paired samples test.

• Formula: Mean of paired differences / Standard error of the mean paired difference compared to 0.

Simon Task

• A classic effect in Psychology.

• Participants are faster at responding to stimuli that appear in a position that is spatially similar to the response that they need to make congruently.

• We now define pairs of variables to specify our test, many of the familiar options for descriptives and assumption checks are available as well.

Simon Task Results

• Participants also repeated the task on another day.

• We can run a second paired samples t-test to see whether the reaction times for the congruent condition changes over time.

• The reaction times do not show a significant difference.

• It is likely that reaction times in this task are reproducible across multiple time points.

Summary

• Two sample hypotheses & study design

  • Comparing the means of two different groups.

  • Using data to answer questions.

• Independent Samples T-Tests

  • How does a two-sample t-test work?

  • What are its assumptions?

  • How can we run this in Jamovi?

• Paired Samples T-Test

  • A test for within subjects designs.

  • (actually, a one-sample t-test in disguise).

Next Steps

P-values and effect sizes.