PSYC3016 Week 6 Comparing two means

Assumptions of independent t-test

1. Dependent variable is at least interval scale

2. normality

3. independence

4. homogeneity of variance

How can we test normality?

• skewness

• kurtosis

• shapiro-wilks test

platykurtic kurtosis

too flat, below zero

mesokurtic kurtosis

almost at zero, pointiness just right

lepokurtic kurtosis

too pointy, above zero

What does it mean is skewness result is more than 2.0?

Indicates problems

shapiro wilks test

measures the extent to which our data differ from a normal distribution

if the p value of the shapiro wilks test is less than 0.05, what does it mean?

data is significantly different than a normal distribution

if the test statistic of the shapiro wilks test is closer to 1, what does that mean?

the data are closer to perfectly normally distributed. The closer to 0, the more it differs from the population

independence

it is assumed observations are independent of one another woth each group and between each group

negative skew

left tail, negative number

positive skew

right tail, positive number

homogeneity of variance

we assume the population standard deviations are the same in each group

what tests measure homogeneity of variance?

• levenes test

• brown-forsythe test

Levene’s test for equal variances

An analysis of various (ANOVA) is performed on the absolute values of deviation scores, where the MEAN is subtracted from each score (no positive or negative signs)

Brown-Forsythe Test for Equal Variances

An Analysis of Variance (ANOVA) is performed on the absolute values of deviation scores, where the MEDIAN is subtracted from each score (no positive or negative signs)

For both tests, what does it mean if the p-value is statistically significant (less than 0.05)

then the variances are statistically different from one another and the assumption is violated.

How do we make a decision about our independent t test?

compare the test statistic to a critical value

standardized effect size

removes the units of the variables in the effect

unstandardized effect size

describe the size of the effect, but the original units remain in the variables

Difference between Glass’ delta and Cohen’s d

only one of the groups standard deviations in the denominator (usually a control group)

measurements of effect size

• Glass’ delta

• Cohen’s d

Nonparametric Test: Mann-Whitney U test

can be used when the normality and/or homogeneity of variance assumptions of the t test are violated

What is the Mann-Whitney U test based on?

ranked scores (ordinal data) rather than raw scores

Drawbacks of Mann-Whitney U test

• can only say which group tended to score higher, but not by how much

• does not have as much statistical power as the t test

What is the effect size of the Mann-Whitney U test

Probability of Superiority (PS)

Probability of Superiority (PS)

If one were to randomly sample one score from each group (x and y), the PS is the probability of the x score being greater than the y score

Benchmarks for interpreting the PS

Equivalent to small Cohen’s d effect size: .56, medium: .64, large: .71.

Rank-Biserial Correlation

correlation coefficient between one nominal variable, one ranked/ordinal variable, direction (positive/negative) and magnitude (-1 to +1)

Benchmarks for interpreting Rank-Biserial Correlation

small: .13, medium: .30, large: .47

statistical power

the probability of correctly rejecting the null hypothesis (H0) when it is false

statistical power should have a value of at least what

0.80

What does this value mean?

we will correctly reject the null hypothesis when we should 80 percent of the time

factors that influence statistical power

• sample size

• effect size in the population

• alpha level

• directionality of test

how does sample size influence statistical power?

the bigger the sample size, the higher the statistical power

How does effect size influence statistical power?

the larger the effect size in the population is, the greater the statistical power

how does choosing an alpha level above 0.5 affect statistical power?

• increases type 1 error rate

• smaller critical value

• decreases type 2 error rate, increasing statistical power

how does choosing a value less than 0.5 affect statistical power?

• decrease type 1 error rate

• larger critical value

• increases type 2 error rate, decreasing statistical power

directionality of test

usually a two-sided test, choosing a one sided test with alpha level .05 does not change type 1 error rate, creates a smaller critical value, and decreases the error rate thus increasing statistical power.