Secondary Data analysis, analysing differences in means (T Test and ANOVA)

Step to successful secondary analysis

Develop the research questions

Identify the data set

  • Online?

  • Offline?

  • Both?

Evaluate the data set

  • Fit

  • Quality

Define the scope of the data set

T-test

  • A t-test is a statistical tool used to compare the means of two groups to determine in a statistically significant difference exists between them

  • Basic principle: To see if the differences in means across two samples are reliably different

  • T value = Variance between / variance within groups

  • P value = Likelihood of value occurring by chance (statistical difference)

Comparing groups using means and variances

  • Note if your scale variable (dependent) is not normally distributed use non parametric T-test equivalent: Mann Whitney

  • In SPSS: analyse menu to non-parametric tests. and select legacy dialogs . 2 independent samples

What if we have ore than 2 groups to compare?

  • An ANOVA (Analysis of Variance) test is a statistical method used to compare the means of 3 of more groups to determine if there’s a statistically significant difference between them.

  • This also tests for differences across groups

ANOVA basic principle example

  • Is there a relationship between ethnicity ans hours spent at home

Explanatory variable: ethnicity (category data)

Dependent variable: hours out of home (ratio)

  • Please note that dependent variable to be tested was non parametric (not normally distributed) we need to use the non parametric version use Kruskal-Wallis test

  • This version tests differences in rank order (rather than means)

  • In the assignment only do ANOVA is you feel confident with it

Levene’s test of equal variance

  • This test is used because some common statistical procedures assume that variances of the populations from which difference samples are drawn are equal

  • Levene’s test accesses this assumption. It tests the null hypothesis that the population variances are equal (called homogeneity of variance)

  • When interpreting Levene’s test, we typically use a significance level of 0.05. If the P value is less than 0.05, we reject the null hypothesis, indicating unequal variances.

  • If Levene’s test is statistically significant, we have violated this
    assumption and would look at the bottom row of figures. In our case, the sig (p) value is below 0.05, so we have violated the assumption (our variances are not equal), so we can look at the bottom row of figures

  • The next thing to look at in this table is the results of the t-test.
    There’s the t-value, df and significance values (these are the main parts of the output). Note, there are 2 sig (p) values. Our hypothesis was 1-sided (because we specified the direction of the difference), so we look at the 1-sided p-value. The 1-sided p-value is 0.001, which is below 0.05, so we can conclude that the difference in onset of criminal career between males and females is statistically significant- we accept our hypothesis and reject accept the null hypothesis.

Mann-Whitney U Test

  • T-tests rely on our DV being normally distributed.

  • This isn’t always the case…

  • We won’t do this now, but run the tests of normality to check this.

  • This means we cannot rely on the results from a t-test and need to use the non- parametric alternative, Mann Whitney U test

Things to remember

  • Independent variables: are categorical (nominal or
    ordinal) and must define two groups (e.g., male/female); (or
    three+ for ANOVA);

  • Dependent variable: that is continuous (interval or ratio scale) and normally distributed (i.e. age);

  • Normal distribution of dependent variation (or do Mann-Whitney
    instead of T Test/ Kruskal Wallis instead of ANOVA);

  • Levenes test for variance across groups: this effects how to
    interpret findings