T Tests Lecture Notes

Quiz Changes

  • Quizzes will change to provide immediate feedback on answers (correct/incorrect) upon submission of the first attempt.
  • To balance this, the number of questions per quiz will increase from 5 to 10.
  • This change will be implemented from next week onwards.

One Sample T-Test

  • Aimed to gauge understanding of the one-sample t-test.
  • Many students indicated they were mostly following along but needed more study.

Course Structure

  • The current and upcoming weeks are information-dense, covering t-tests (this week), t-tests in labs (next week), ANOVAs (lecture the week after), and ANOVAs in labs (the week after that).
  • This content is crucial for the upcoming assignment.
  • The course will then briefly cover other statistical tests and concepts before returning to assignment content.

General Course Questions

  • Quiz questions cannot be viewed before the lab session.
  • Screenshots can be taken during the quiz for later study.
  • Past quiz questions will be reviewed in the following Tuesday lecture.

T-Tests

  • The one-sample t-test involves a single, continuous variable to determine if its mean differs from an expected value.
  • Today's focus: t-tests for comparing two groups.
    • Paired Samples t-test: Comparing the same variable measured at two time points in the same individuals.
    • Independent Samples t-test: Comparing the same variable measured in two different groups.
  • More than two groups require ANOVAs (Analysis of Variance).
  • Correlation and Regression will be covered after t-tests and ANOVAs but will not be part of this year's assignment.

Paired Sample T-Test ( repeated measures )

  • Also known as repeated measures t-test.
  • Compares the means of two groups when the data comes from the same individuals (almost always).
    • Example: Measuring statistics knowledge at the start and end of a semester.
  • Requires one continuous, quantitative outcome measure and a grouping variable with two levels (a binary categorical variable).
    • Example: Statistics score (continuous) and time point (pre/post course - categorical).
  • Time point (pre/post) is the predictor variable, while statistics score is the outcome measure.
  • Asks whether there's a difference between the two groups.
  • Transforms the measured mean and standard deviation into a T statistic.
  • The T statistic follows a T distribution, which helps determine the probability of the observed change in means under the null hypothesis.
  • Terminology: Repeated measures and paired samples are interchangeable.

Research Scenario: Children's sleep duration

  • Question: Do three-year-old children sleep less than one-year-old children?
  • Recruit 60 children at age one, measure their average nightly sleep duration for a week.
  • Revisit the same children at age three and repeat the measurement.
Hypotheses:
  • Alternative Hypothesis: Three-year-olds sleep less than one-year-olds (directional hypothesis).
  • Null Hypothesis: Three-year-olds sleep more or the same amount as one-year-olds.
Power Analysis
  • A power analysis can determine the necessary number of participants, the chances of detecting a reasonable effect size, or the smallest detectable effect size, given the resources.
  • Example: With 60 participants, determine the smallest effect size detectable with 80% power.
  • R may provide negative values for the effect size, depending on the phrasing of the alternative hypothesis (whether values are expected to be less than or higher than).
Code
  • The type of t-test (paired) and the alternative hypothesis (directional) should be specified in the power analysis.
  • Specifying alternative = "less" indicates a directional hypothesis.
  • If the alternative hypothesis is rephrased (e.g., one-year-olds sleep more than three-year-olds), the code should reflect this change (e.g., alternative = "greater").
  • The order of variables in the t.test function is crucial and should align with the phrasing of the alternative hypothesis.
Effect Size
  • An effect size of 0.32 is considered a moderate effect.
  • Power analysis indicates the sensitivity of the test, showing whether the number of participants is sufficient to detect an effect size of interest.
  • N refers to the number of pairs in this context, emphasizing that 60 represents the number of children measured at two time points.
Data Collection and Descriptive Statistics
  • Data is entered into a spreadsheet with participant number, hours slept at one year old, and hours slept at three years old.
  • Calculate and report descriptive statistics (mean and standard deviation (σ)(\sigma)).
  • Plotting data can provide an intuitive visual assessment of potential differences.
Assumptions
  • Measurements are from a random sample.
  • Observations are dependent on one another (taken from the same people at different time points).
  • Difference values should be normally distributed.
Normality
  • For one sample t-tests, the values should be normally distributed.
  • For repeated measures, the differences between the first and second measurements should be normally distributed.
  • Use a histogram to visualize the distribution of differences.
  • Use the Shapiro-Wilk test to assess if the data significantly deviates from normality.
Shapiro-Wilk Test
  • Aim for a non-significant result, indicating no significant difference between the distribution and a normal distribution.
  • If the test is significant, the data significantly differs from a normal distribution.
Test Statistic and Degrees of freedom
  • Optional info: test statistic arrived at test and degrees of freedom. Won't be asked about in quizzes or exams.
  • Formula for the repeated measures T statistic:
    t=mean differencestandard deviation of the differencenumber of observationst = \frac{\text{mean difference}}{\frac{\text{standard deviation of the difference}}{\sqrt{\text{number of observations}}}}
  • Under the null hypothesis, the T statistic follows a T distribution with degrees of freedom (df) = n - 1 (where n is the number of participants).
  • As sample size increases, the T distribution converges on a normal distribution.
  • Different shapes of the distribution are due to different degrees of freedom which captures how probable extreme values are based on sample size.
One-Tailed Test
  • The hypothesis indicated a one-tailed test since it was specifically interested in whether the means mean hours slept at three year old three years old is lower than one years old.
  • Using (α=0.05)(\alpha = 0.05), the whole 5% is put in lower tail end of the distribution.
P-Value
  • Very small number, R gives the value in scientific notation:
    • 2e-16, equivalent to 2×10−162 \times 10^{-16}. Decimal place moved backwards 16 places.
  • Significant effect, so reject the null hypothesis.
Decisions
  • Rejecting or retaining the null hypothesis means we're choosing which column of a decision table we're in.
  • But we don't know which row we're in, since it's whether there really is a difference or not. We can't ever fundamentally know that.
  • Significant result means rejecting it, and accepting that the three year old mean is less than the one year old.
  • False positive has an error probability of 5% (α)(\alpha).
T-Test in R
  • Run repeated measures t-tests in R, using: t.test(variable1, variable2, paired = TRUE, alternative = ...)
  • The R output provides the test that was run, the T value, degrees of freedom, and P value.
Effect Size
  • Effect sizes are calculated for significant effects, using cohensD(data, var.1, var.2, method = "paired")
Report Writing
  • Consult the APA guide for reporting numbers and statistics (linked on Canvas).
  • Begin with descriptive statistics (mean and standard deviation) for both groups.
  • State the type of statistical test and its result, mentioning if it's a significant, one-tailed, repeated measures t-test.
  • Include the test statistic (T value), degrees of freedom, and P value in brackets: t(df) = t-value, p = ....
  • Report the P value to two or three decimal places, but state it as p < 0.001 if it's very small.
  • If the result is significant, calculate and state the effect size (Cohen's d).
  • Example: The same children, when measured at three years old, slept fewer hours per day (M = x, SD = y) than they had at one year old (M = a, SD = b), confirmed by a significant, one-tailed, repeated measures t-test, t(59) = -11.51, p < 0.001. This reduction in sleep was a large effect size: Cohen’s d = 1.03.

Independent Samples T-Test

  • Compares the means of two groups when the data comes from two independent separate groups of people. Experimental difference determines whether you use repeated/paired or independent!
    • Again, you're going to have one continuous quantitative outcome measure, and 2 categories, a categorical predictor that has two levels/groups. Asks if there is a mean difference between the two groups
    • Still going to use a test statistic. Is still the T statistic which is still going to follow a T distribution!
Research Scenario: Vocabulary Size in Children
  • Comparing vocabulary size between children in monolingual and multilingual households.
  • Recruit children from both types of households and measure their vocabulary size.
Research Question and Hypotheses

*Research question question here is phrased, Is there a difference? which implies Non directional.
*Alternative Hypothesis: Means in the monolingual group is merely different from the means in the multilingual group. The null hypothesis merely has to be everything other than that, which is nice and simple. That they're equal

Power Analysis
  • Assuming 60 children per group, perform a power analysis to check the sensitivity to detect a reasonable effect.
Code Implementation

Alternative two-sided has to be specified because it is a non directional two tailed hypothesis. Otherwise, has to specify exactly what kind of test is intended for accurate output.

  • D = 0.5. So that's a moderate effect size it could detect.

  • With less sensitivity, we'd still be happy to run the test.

Data Structure

  • Data Structure: Participant Number, number of words/vocabulary size, Categorical variable called "group" tells it from which household.

  • Both of these are valid ways to structure data where you have measurements from two groups

Descriptive statistics

  • Summary and descriptive statistics should be gathered.

  • This should be for both monolingual and multilingual groups.

Assumptions

  • The sampling assumptions are a consideration as always.
    Sampling assumptions should be checked as best as one can. Randomly sample? Bias?

  • The observations are independent for one another. So 2 groups are just separate people. They can't influence one another in an obvious type of way

  • Then there's the normality assumption. And here, the independent samples t test assumes that both of our measurements separately are normally distributed
    So normality is checked for both groups.

  • We could have a look at the histograms, but what we really need to do is run a Shapiro Wilk test
    And that'll tell us if they deviate significantly from normality.
    We hope that both of them are not significant, which is what we find here. P equals 0.5, p equals 0.1.

*There is another consideration to keep in mind:
It is often said that an independent sample c test assumes that the variability in each of your groups is the same as one another (standard deviation similar). HOWEVER - R involves a correction. Involves a way of sort of combining the SD in a way that doesnt assume that they're equal

Calculations of T value

  • R is calculating the T value from the data:

  • t=difference between means of the two groupsstandard error of the differences between the two groupst = \frac{\text{difference between means of the two groups}}{\text{standard error of the differences between the two groups}}

  • It finds the corresponding T distribution that the value is going to follow and calculates how probable that T value would be, finding the P value.

  • Concluding the P value by comparing the distributions, the probability is calculated as whether the null hypothesis can be rejected
    Hopefully, we've made a true positive decision, a correct decision. So we've concluded that children from multilingual households have a bigger vocabulary than children from monolingual households. Although our hypothesis is directional, but we can still talk about directionality in our effect because we can look at our means, and we see that the multilingual is bigger than the monolingual, and there's a significant difference.

How to run in R
  • The user should use: t.test with arguments specific to the variables used. Alternative must be specified in this function
Effect Size
  • If there's a significant effect, effect size can also be calculated. D = Difference / SD. Cohen function and method are used to specify the value
  • Method unequal should be used
  • Is it a problem or a contradiction if your observed effect size is smaller than the one that you had sufficient power to detect, provided the effect is significant. You don't have any reason to think of it as furious.
Reports
  • Structuring Reports: Descriptive - Results - Outcome of test

Assumptions and Nonparametric Tests

  • All the flavors of T Test make some type of assumption.

  • Common statistical tests assume that your data are normally distributed

  • Any test that assumes that your data follows a certain kind of distribution is called a parametric test. The logic behind that is that the normal distribution is controlled by just a couple of parameters.

  • Often, that works quite well. I sort of mentioned last week, I think this idea that lots of things in nature do follow normal distributions because it's just the shape that arises if you have lots of different influences on something. You know, people's heights and weights and so on tend to be normally distributed just because they're influenced by lots of independent things

Nonparametric Tests

*Tests do have assumptions. parametric tests assume that your data follow a certain distribution, if that's not met, you can still generally find a statistical test that you can use, but you have to go figure out what the nonparametric alternative is for the test you had hoped to use. *You can use them all of the time.* Generally, the cons are that they'll tend to have a lower power. tend to be less sensitive

### R and non parametic tests

  • Significant Shapiro? Non-primal test
    When we look at assumptions, and significant shapiro, it has to be not normal in order to look at non primal test. if our assumptions are met, we try to use the parametric versions if we can but if we cant, there parametric alternatives.

  • Alternatives: Wilcoxon(T test) and Kruskal-Wallis test(ANOVA)

  • Parametric or not -
    *sampling assumptions.

Sampling assumption must be true whatever happens. no alternative test.Premise of ALL statistical testing is random sampling.

What to do if Assumptions are Not Met
  • If normality is not normally distributed, The nonparametric is the Wilcoxon signed rank test.
  • This would mean to change the t to wilcox code.
  • Sign rank/Rank Sum for both groups, R does not handle this so look for another package.