T-Test Notes

t-test

Review of 6 Steps for Significance Testing

Step 1: Set alpha (p-level).
Step 2: State hypotheses: Null and Alternative.
Step 3: Calculate the test statistic (sample value).
Step 4: Find the critical value of the statistic.
Step 5: State the decision rule.
Step 6: State the conclusion.

t-test

The t-test is about means: distribution and evaluation for group distribution.
It's derived from the normal distribution.
The shape of the distribution depends on sample size; the sum of all distributions is a normal distribution.
The t-distribution is based on sample size and varies according to the degrees of freedom.

What is the t-test?

The t-test is a useful technique for comparing mean values of two sets of numbers.
The comparison provides a statistic for evaluating whether the difference between two means is statistically significant.
The t-test is named after its inventor, William Gosset, who published under the pseudonym "student."
The t-test can be used either:
- To compare two independent groups (independent-samples t-test).
- To compare observations from two measurement occasions for the same group (paired-samples t-test).

What is the t-test? (Null Hypothesis)

The null hypothesis states that any difference between the two means is a result of a difference in distribution.
It assumes that both samples are drawn randomly from the same population.
It compares the chance of having a difference in one group due to a difference in distribution.
The assumption is that if both distributions came from the same population, both distributions have to be equal.

What is the t-test? (Difference due to Chance)

The intent is to find the difference due to chance.
Logically, the larger the difference in means, the more likely to find a significant t-test.
Recall:
- Variability: More variability = less overlap = larger difference.
- Sample size: Larger sample size = less variability (pop) = larger difference.

Types of t-tests

The independent-sample t-test:
- Used to compare two groups' scores on the same variable.
- For example, it could be used to compare the salaries of dentists and physicians to evaluate whether there is a difference in their salaries.
The paired-sample t-test:
- Used to compare the means of two variables within a single group.
- For example, it could be used to see if there is a statistically significant difference between starting salaries and current salaries among the general physicians in an organization.

Assumptions of t-tests

The dependent variable should be continuous (Interval/Ratio scale).
The groups should be randomly drawn from normally distributed and independent populations
- e.g., Male vs. Female, Dentist vs. Physician, Manager vs. Staff (NO OVERLAP).
The independent variable is categorical with two levels.
Distribution for the two independent variables is normal.
Equal variance (homogeneity of variance).
Large variation = less likely to have a significant t-test = accepting the null hypothesis (fail to reject) = Type II error = a threat to power
- Analogized as sending an innocent to jail for no significant reason.

Independent Samples t-test

Used when we have two independent samples, e.g., treatment and control groups.
Formula:
t = \frac{\bar{X1} - \bar{X2}}{SE{\bar{X1} - \bar{X_2}}}
- Terms in the numerator are the sample means.
- Term in the denominator is the standard error of the difference between means.

Independent samples t-test (Standard Error Formula)

The formula for the standard error of the difference in means:

SE{diff} = \sqrt{\frac{SD1^2}{N1} + \frac{SD2^2}{N_2}}

Example scenario: Studying the effect of caffeine on a motor test where the task is to keep the mouse centered on a moving dot. Half get caffeine, half get a placebo; nobody knows who got what.

Independent Sample Data

Data are time off task.
Experimental (Caffeine):
- 12, 14, 10, 8, 16, 5, 3, 9, 11. N1=9, M1=9.778, SD1=4.1164
Control (No Caffeine):
- 21, 18, 14, 20, 11, 19, 8, 12, 13, 15. N2=10, M2=15.1, SD2=4.2805

Independent Sample Steps (1)

Set alpha.
- Alpha = 0.05
State Hypotheses.
- Null hypothesis: H0: \mu1 = \mu_2
- Alternative hypothesis: H1: \mu1 \neq \mu_2

Independent Sample Steps (2)

Calculate test statistic:

SE_{diff} = \sqrt{\frac{4.1164^2}{9} + \frac{4.2805^2}{10}} = \sqrt{\frac{10(4.2805)^2 + 9(4.1164)^2}{}} = 1.93

t = \frac{9.778 - 15.1}{1.93} = \frac{-5.322}{1.93} = -2.758

Independent Sample Steps (3)

Determine the critical value.
- Alpha is 0.05, 2 tails, and df = N1+N2-2 or 10+9-2 = 17. The value is 2.11.
State decision rule.
- If |-2.758| > 2.11, then reject the null.
Conclusion:
- Reject the null. The population means are different. Caffeine has an effect on the motor pursuit task.

Table 4: Percentage Points of the t-distribution

(A table showing critical t-values for various degrees of freedom and alpha levels is shown.)

Using SPSS

Open SPSS
Open file "SPSS Examples" for Lab 5
Go to:
- "Analyze" then "Compare Means"
- Choose "Independent samples t-test"
- Put IV in "grouping variable" and DV in "test variable" box.
- Define grouping variable numbers.
- E.g., the experimental group is labeled as "1" in the data set and the control group as "2"

Independent Samples Exercise

Perform the t-test by hand and with SPSS.
Experimental: 12, 14, 10, 8, 16
Control: 20, 18, 14, 20

SPSS Results (Example Output)

Group Statistics table shows descriptive statistics for each group (experimental and control).
Independent Samples Test table shows:
- Levene's Test for Equality of Variances (F, Sig.)
- t-test for Equality of Means (t, df, Sig. (2-tailed), Mean Difference, Std. Error Difference, 95% Confidence Interval of the Difference)

Dependent Samples t-tests

Dependent Samples t-test

Used when we have dependent samples – matched, paired, or tied somehow
- Examples: Repeated measures, Brother & sister, husband & wife, Left hand, right hand, etc.
Useful to control individual differences. Can result in a more powerful test than the independent samples t-test.

Dependent Samples t Formulas

t is the difference in means over a standard error.
The standard error is found by finding the difference between each pair of observations. The standard deviation of these differences is SD_D.
Divide SDD by sqrt (number of pairs) to get SE{diff}.

t = \frac{\bar{X}D}{SED}

SE{diff} = \frac{SDD}{\sqrt{n_{pairs}}}

Another way to write the formula

t = \frac{\bar{X}D}{\frac{SDD}{\sqrt{n_{pairs}}}}

Dependent Samples t example

(A table is shown with the following data)

Person	Painfree (time in sec)	Placebo	Difference
1	60	55	5
2	35	20	15
3	70	60	10
4	50	45	5
5	60	60	0
M	55	48	7
SD	13.23	16.81	5.70

Dependent Samples t Example (2)

Set alpha = 0.05
Null hypothesis: H0: \mu1 = \mu2. Alternative is H1: \mu1 \neq \mu2.
Calculate the test statistic:

SE = \frac{SD}{\sqrt{n}} = \frac{5.70}{\sqrt{5}} = 2.55

t = \frac{\bar{X}D}{SE{diff}} = \frac{55 - 48}{2.55} = \frac{7}{2.55} = 2.75

Dependent Samples t Example (3)

Determine the critical value of t.
- Alpha = 0.05, tails=2, df = N(pairs)-1 = 5-1 = 4. Critical value is 2.776
Decision rule:
- Is absolute value of the sample value larger than the critical value?
Conclusion:
- Not (quite) significant. Painfree does not have an effect.

Using SPSS for dependent t-test

Open SPSS
Open file “SPSS Examples” (same as before)
Go to:
- “Analyze” then “Compare Means”
- Choose “Paired samples t-test”
- Choose the two IV conditions you are comparing. Put in “paired variables box.”

Dependent t- SPSS output

Paired Samples Statistics table shows descriptive statistics for each variable (e.g., PAINFREE and PLACEBO).
Paired Samples Correlations table shows the correlation between the two variables.
Paired Samples Test table shows the t-test results, including:
- Mean, Std. Deviation, Std. Error Mean, 95% Confidence Interval of the Difference, t, df, Sig. (2-tailed).

Relationship between t Statistic and Power

To increase power:
- Increase the difference between the means.
- Reduce the variance
- Increase N
- Increase α from α = 0.01 to α = 0.05

To Increase Power

Increase alpha, Power for α = .10 is greater than power for α = .05
Increase the difference between means.
Decrease the sd’s of the groups.
Increase N.

Independent t-Test with SPSS: Data Input

Show the steps to perform an independent t-Test with SPSS

Independent t-Test: Independent & Dependent Variables

Show the steps to select variables

Independent t-Test: Define Groups

Show the steps to define groups after selecting variables

Independent t-Test: Options

Show the option menu to check the confidence interval

Independent t-Test: Output

Output: Are the groups different? No difference found.
Example: t(18) = .511, p = .615

Dependent or Paired t-Test: Define Variables

Show data input of pre and post data sets

Dependent or Paired t-Test: Select Paired-Samples

Show how to select Paired-Samples t-test from compare means menu

Dependent or Paired t-Test: Select Variables

Show how to select paired-Sample variables into correct box

Dependent or Paired t-Test: Options

Show the option of defining confident Interval

Dependent or Paired t-Test: Output

Is there a difference between pre & post? Yes
t(9) = -4.881, p = .001
4. 7 is significantly different from 6.2