STATS FINAL: T-tests and Mann-Whitney U test (Wilcoxon Rank)

What is a student’s T-test?

It’s the same thing as a T-test

The Student's t-test (or simply the t-test) is an inferential statistical test used to determine if there is a statistically significant difference between the means of two groups.

The T-test is a statistical tool that takes the difference between the averages of two groups, calculates a T-value, and then compares that T-value against a special "t-curve" to figure out if the difference you saw is a real effect or just random luck.

Who developed the T-test?

William Sealy Gosset ==> developed the T-stat

T-test vs. Z-test:

The population standard deviation is unknown, it’s estimated from the sample instead

What does the T-distribution look like?

The t-distribution is flatter and has heavier tails than the normal (Z) distribution

• when the sample size gets larger, that’s when the distribution starts to look relatively similar to the Z.

What is a Test Statistic?

Test Statistic is the single, crucial number you calculate from your data to see if your results are significant (helps us determine P value = aka. significance level)

• A test statistic summarizes your observed data into a single number.

• Think of it as summarizing all the information about your sample (like the average, the variability, and the size) into one number.

Test statistics differ depending on the statistical test we’re using:

uA test statistic follows a certain distribution.

What are the steps to calculating a test statistic?

The test statistic is calculated right in the middle of your statistical process:

1. Formulate hypotheses (decide what you're testing: H0 and H1).

2. Choose the right statistical test (e.g., T-test, ANOVA).

3. Calculate the test statistic corresponding to that test.

4. Determine the p-value (based on where your calculated test statistic falls on the distribution curve).

A larger T-value from the T-test means:

• X1 and X2 are the sample means

• N1 and N2 are the sample sizes for each groups

• Sp2 is pooled variance

A larger difference in the numerator means a larger t-value, which suggests a stronger case for a significant result.

The bigger the T-stat, the better because:

The t-statistic is your metric for convincing yourself (and others) that the difference you found is a true signal and not just statistical noise.

**The bigger the t-value, the more compelling your evidence is that the two groups are different

What is the difference between the 3 different T-tests:

One sample T-test: comparing population mean to sample mean

Example: Comparing the mean of a single sample to a known value or a theoretical mean

• (Does the average height of students in your class differ significantly from the national average height of 5'7"?

Two sample indepedant T-test: Classic

Purpose: Compares the means of two separate, independent groups.

e.g., comparing the test scores of two different groups of students

Paired/dependant T-test:

Purpose: Compares the means of two related or paired groups

(e.g., comparing students' scores before and after taking a prep course).

T-tests difference simplified:

One sample ==> testing sample mean to a known “popular mean”

Two-sample (independant) ==> comparing the mean between 2 indepedant groups

Paired T-test ==> comparing the mean between 2 dependant/paired groups (pre and post)

How do we determine if an indepedant t-test is appropriate?

1. Assumption of independence

2. Dependant/responding variable MUST BE continuous (interval or ratio)

3. The manipulated/indepedant variable must be binary categorical (women vs. men, treatment vs. control)

*so basically the manipulated variable must be a binary category and the responding variable must be continuous like blood pressure between men vs. women

What are the assumptions for conducting a two-sample indepedant T-test?

1. random representative sample

2. continuous DV

3. independant observation

4. normality (normal distribution)

5. Homogeneity of Variance (equal variance) ==> the spread should be around the same

Homogeneity/Equal Variance means the spread of data should be around the same, and if you were to draw a distribution curve (like a bell curve) for each of your two independent groups, they would look similar in their width (or flatness) .

How do we check normality and equal variance for an indepedant T-test, like checking the assumptions before we input datat?

Normality is tested ==> via. Shapiro-wilk test

Equal variance is tested ==> via. Levene’s Test

What do we use if the variance’s are unequal for an indepedant T-test?

Welch’s T-test ==> like the gummy bears.

I hate gummy bears and so we want a homogeneity of variance to avoid Welch’s

What are degrees of freedom and what’s the difference between small vs. large df (degrees of freedom)

Degrees of freedom (df) are essentially the number of values in a final calculation that are free to vary.

• Small df: The T-distribution is wider and shorter (fatter tails), reflecting greater uncertainty because the sample size is small.

• Large df: The T-distribution becomes narrower and looks almost exactly like the normal Z-distribution. This happens as the sample size gets large, and your estimate of the population standard deviation becomes more reliable.

How do we determine P value for an independent T-test?

1. degrees of freedom

2. determine critical value and rejection regions via. T-distribution table

3. Compare T-stat to critical value

What is critical value?

A critical value is the value of the test statistic that defines the Rejection Region (or Critical Region).

1. The Simple Concept: The "Line in the Sand"

Think of the critical value as the boundary line drawn on the distribution curve that separates results that are considered common from results that are considered rare (and therefore statistically significant).

When does a T-value fall into the rejection region?

When the T-value is more extreme than the critical value = there IS a significance = reject null

Conclusion: Your result is rare enough to conclude that it did not happen by random chance under the null hypothesis. You reject the null hypothesis (H0).

How do we find critical value?

Based on the degrees of freedom we calculate

the # of data -1 in group A and B added

If the critical value isn’t available in the table we:

round down

So we look at:

1. whether we’re doing one-tail or two-tail

2. we look for our alpha value/significance levels

3. Go down to the degrees of freedom

AND THAT’S OUR CRITICAL VALUE, SO IF WE PASS IT WE REJECT NULL

Two-tailed vs. One-tailed

1. Two-Tailed Test (Non-Directional)

• When to Use It: Use a two-tailed test when you are simply asking, "Is there a difference?"

  • You are open to the possibility that the average of Group 1 could be higher OR lower than the average of Group 2.

2. One-Tailed Test (Directional)

• When to Use It: Use a one-tailed test when you are asking, "Is one group significantly greater than (or less than) the other?" You are only looking for a difference in one specific, predicted direction.

Be cautious when selecting a one-tailed test because:

One-tailed tests have more power to detect differences, but also more like to make Type I error (false positive).

What are the characteristics of a Mann-Whiteny U test (Wilcox Rank-sum test)

The Mann-Whitney U test (or Wilcoxon Rank-Sum test) is the non-parametric alternative to the Independent Samples T-test.

• Do not assume normality, this is for skewed data or data with outliers

• Ranks instead of means, so the test ranks all the values from both groups together and compares a sum between the group

• Works with ordinal and continuous data

Independant observations still apply for the Wilcoxon Rank-Sum test, yes or no?

YES

The assumption of independent observations still apply: the two groups being compared should be independent (no repeated measures, paired data, or matched samples).

What statistic do we calculate for the Wilcoxon rank-sum/Mann U whitney?

U-statistic

H₀: The distributions (ranked sum) of the two groups are equal.

H_a: The distributions (ranked sum) of the two groups are not equal (for a two-tailed test).

What are paired T-tests used for?

comparing the means (responding variable) of 2 groups that are dependant

Longitudinal: repeated measures from same individuals over time

Cross-sectional: data from pairs of people that are likely to be similar for reasons such as genetics (twins), home environment (married couples, patient-family caregiver dyads )

What are the assumptions of paired T-tests?

• Subjects are presumed to be randomly sampled (representative sample)

• Continuous DV and Binary IV

• Normality ==> The differences between paired values are approximately normally distributed.

What would the null and alternative hypothesis look like for a paired T-test?

Null = no difference pre- and post treatment

Alternative = Pre and post not equal (two-tailed test)

What is the non-parametric to a paired T-test?

Wilcoxon SIGNED rank test

• no normality

• ranking instead of mean difference

• ordinal and continuous data

How do we determine if there is no difference between pre and post treatment (paired T-test)?

The average difference between the two groups is = zero

so we do post minus pre and then take the average of those points

For the wilcoxon-signed rank test, how do we determine mean difference?

The median is the appropriate measure of the center.

The null hypothesis for the Wilcoxon Signed-Rank test is that the median of the population difference scores is zero.

When can we not do a wilcoxon signed rank test, even with pairs?

When there are less than 5 pairs, so we only have 5 people per group basically