Week 11 – Statistical Tests, Chi-Square & Odds Ratios

Parametric tests – what they are and their typical assumptions

The statistical tests covered earlier in the unit; they rely on assumptions being true | Typically: DV is interval or ratio; DV is normally distributed; each participant took part only once

What is nominal data?

Data where responses are categories or classes. Examples: male/female, depressed/not depressed, religion. Categories can be numerically coded (e.g. male = 1; female = 2), but the numbers themselves are not meaningful.

What is ordinal data?

Data where responses are ordered on a continuum, but the intervals between them are not necessarily equal. Examples: unit grades (P, C, D, HD) and race outcomes (1st, 2nd, 3rd).

What is interval data?

Quantitative data with equal intervals between values but no true zero; zero does not mean the absence of what is measured. Example: scores on a personality scale.

What is ratio data?

Like interval data, but with a true zero value. Examples: height, weight and speed.

When are nonparametric tests used?

When you have ordinal or nominal data, OR interval/ratio data that isn't normally distributed (or you aren't confident it is) | If the assumptions don't hold you can't be confident parametric tests will give accurate answers

Nonparametric tests – other name and assumptions

Also called distribution-free tests | They still have some assumptions, but much less restrictive than those of parametric tests

What nonparametric tests are listed in Lecture 11?

Chi-square goodness of fit; chi-square test of independence; Mann-Whitney U test; Wilcoxon’s signed ranks test; Kruskal-Wallis test.

Choosing a test – key principle and power comparison

Generally use the most powerful test that is appropriate to the data | Parametric tests tend to be more powerful than distribution-free ones (more likely to reveal a significant effect), provided their assumptions hold

Test-selection flowchart – colour key and the questions it asks

Green = parametric tests; red = nonparametric tests | Questions asked: type of DV (continuous interval/ratio vs categorical nominal/ordinal); how many IVs; type of IV (continuous/categorical); how many categories; between or within subjects; population SD known?

Flowchart – continuous DV, no IV: which test?

Population SD known → z test (sample) | Population SD unknown → one-sample t-test

Flowchart – continuous DV, one continuous IV: which tests?

Pearson's r correlation (parametric) | Kendall's tau (nonparametric)

Flowchart – continuous DV, one categorical IV with TWO categories: which tests?

Between subjects: independent-groups t-test (parametric) / Mann-Whitney U test (nonparametric) | Within subjects: paired-samples t-test (parametric) / Wilcoxon rank test (nonparametric)

Flowchart – continuous DV, one categorical IV with MORE THAN TWO categories: which tests?

Between subjects: one-way ANOVA (parametric) / Kruskal-Wallis test (nonparametric) | Within subjects: repeated-measures ANOVA (parametric) / Friedman's test (nonparametric)

Flowchart – continuous DV, TWO OR MORE IVs: which test?

Continuous IVs: multiple regression | Categorical IVs: between-groups → factorial between-groups ANOVA; within-groups → factorial within-groups ANOVA; between-and-within → mixed ANOVA

Flowchart – CATEGORICAL DV (nominal/ordinal): which test?

Zero IVs: chi-square goodness of fit | One continuous IV: logistic regression (green/parametric on the flowchart) | One categorical IV: chi-square test of independence (both chi-square tests are nonparametric)

What is chi-square (χ²)?

A nonparametric test for categorical data (nominal or ordinal – the categories may be ordered or unordered) | Analyses the frequencies within categories | Symbol χ², pronounced 'kie-square'

What does chi-square compare, and what does the χ² statistic express?

Compares observed (obtained) frequencies with expected frequencies | The χ² statistic expresses the discrepancy between them | Tests whether the frequencies observed in the categories differ from what we would expect in the population

Chi-square formula – and what O and E mean

χ² = Σ[(O − E)² / E] | O = observed frequencies (measured/counted in your study) | E = expected frequencies (based on knowledge of the population or a theoretical perspective; how you calculate them depends on goodness of fit vs test of independence)

Chi-square distribution – key features

A known probability distribution (like the t-test) whose shape depends on N | Different curves for different degrees of freedom | The df reflects the number of independent categories

Chi-square GOODNESS OF FIT – when is it used and what does it measure?

Used with a single variable with two or more categories (e.g., course: Science/Arts/Law/Engineering; pizza preference: meat lovers/vegetarian/supreme/chicken) | Tests whether observed frequencies fit the pattern you expect – a measure of the degree of fit between observed and expected frequencies

Chi-square goodness of fit – H0 and what rejecting/retaining it means

H0: the observed data fit the expected data | Reject H0 → the data do not fit the expected data/theory | Retain H0 → the actual data fit the expected data, so the theory is supported

What is the degrees-of-freedom formula for chi-square goodness of fit?

df = number of categories − 1.

Goodness of fit worked example 1 (hiring) – set-up, hypotheses, observed and expected frequencies

120 people hired in the past year; equal numbers of males and females applied; no difference in hiring policy would predict 50:50 | Research hypothesis: numbers of males and females employed differ | H0: the numbers are the same; H1: the numbers differ | Observed (counted): 75 females, 45 males | Expected (predicted by theory): 60 females, 60 males

Goodness of fit worked example 1 (hiring) – calculation, result and conclusion

χ² = (75−60)²/60 + (45−60)²/60 = 3.75 + 3.75 = 7.50 | df = 1 (categories − 1); N = 120; slide reports p = .034 → significant (jamovi reports χ² and p for you) | Females were chosen more often than males; observed ≠ expected, so it is highly unlikely there is no discrimination in hiring at this company | (Check: χ² = 7.50 with df = 1 actually gives p ≈ .006 – the slide's .034 is just an 'it might report' figure; the conclusion is the same)

What is the recommended write-up for the hiring goodness-of-fit example?

“The chi-square goodness of fit test was used to examine whether gender was a factor in hiring practices at a company. If gender was not a factor in hiring there would be an equal number of males and females hired. The chi-square test was statistically significant (χ²(1, N = 120) = 7.50, p = .034), indicating that females were chosen more often than males.”

Goodness of fit worked example 2 (ACT physical activity) – set-up, observed and expected

Question: does the ACT differ from the general population in the proportion engaging in sufficient physical activity? Previous research: about 25% of the general population do | 400 ACT people surveyed – observed: 254 don't engage, 146 do | Expected: 300 don't, 100 do (25% of 400 = 100) – a 3:1 ratio (75%:25%)

Goodness of fit – key points from the ACT example (expected frequencies)

Calculated exactly like the hiring example (same formula); only the expected frequencies differ | Expected frequencies need not be equal – they can follow a theoretical pattern (e.g., 3:1 or 75%:25%) | Total observed must equal total expected (400 in this example)

Chi-square TEST OF INDEPENDENCE – when is it used and what does it test?

Used with two variables, each with two or more categories (e.g., age

Chi-square test of independence – H0 and what rejecting/retaining it means

H0: the two variables are independent | Reject H0 → the variables are not independent; they are related/associated (e.g., political preference depends on gender) | Retain H0 → independent in the population; knowing one variable tells you nothing about the other

Independence worked example 1 (voting) – set-up and observed frequencies

140 people reported their gender and who they would vote for (two-party preferred); variables: political preference and gender | Observed: Females – 59 Labor, 29 Coalition; Males – 15 Labor, 37 Coalition

Chi-square test of independence – expected frequency formula (with the voting example)

E = (row total × column total) / N, calculated for each cell | Voting: row totals F 88, M 52; column totals Labor 74, Coalition 66; N = 140 | e.g., females voting Labor: E = (88 × 74)/140 = 46.51

Independence worked example 1 (voting) – expected frequencies, calculation, result and conclusion

Expected: Females – Labor 46.51, Coalition 41.49; Males – Labor 27.49, Coalition 24.51 | χ² = (59−46.51)²/46.51 + (29−41.49)²/41.49 + (15−27.49)²/27.49 + (37−24.51)²/24.51 = 3.35 + 3.76 + 5.67 + 6.36 = 19.14 (the slide lists the 2nd and 3rd terms in the other order; the total is the same) | Slide reports p = .043 < .05 → reject H0 | Political preference is influenced by gender (related, not independent); females are significantly more likely than males to vote Labor | (Check: χ² = 19.14 with df = 1 actually gives p < .001 – the slide's .043 doesn't match; the conclusion is the same)

What is the APA-style write-up for the voting example?

“The chi-square test of independence was used to examine whether political preference is related to gender. The chi-square test was statistically significant (χ²(1, N = 140) = 19.14, p = .043), indicating that there is an association between gender and political preference.”

Chi-square test of independence – degrees of freedom formula

df = (r − 1) × (c − 1), where r and c are the numbers of levels of the two categorical variables | e.g., gender (2) × hair colour (3): df = 1 × 2 = 2 | Voting example: 2 × 2 levels → df = 1

Chi-square test of independence – conceptually, what is it testing?

Whether there is an interaction between two variables: the effect of a level of one variable depends on which level of the other you are talking about | e.g., the proportion of people who vote Labor depends on whether the person is male or female

Independence worked example 2 (violent TV and fighting) – set-up and observed frequencies

Question: is preference for violent vs non-violent TV related to a history of fighting at school? H0: fighting at school is not dependent on TV preference | Observed: Prefer violent TV – 40 history of fighting, 15 no history (row total 55); Do not prefer violent TV – 30 history, 70 no history (row total 100) | Column totals: history 70, no history 85; N = 155

Independence worked example 2 (violent TV and fighting) – expected frequencies, calculation, result and conclusion

Expected: Prefer violent TV – 24.84 (history), 30.16 (no history); Do not prefer – 45.16 (history), 54.84 (no history) | χ² = (40−24.84)²/24.84 + (15−30.16)²/30.16 + (30−45.16)²/45.16 + (70−54.84)²/54.84 = 9.25 + 7.62 + 5.09 + 4.19 ≈ 26.15 (unrounded 26.16, which is what the slide's write-up uses) | p < .001 → reject H0 (the samples differ too much to have come from the population defined by H0) | There is a relationship between preferred TV program and fighting behaviour

Odds ratio (OR) – what is it?

The odds that an outcome occurs given an exposure, compared with the odds of the outcome without that exposure | A relative measure of effect (compares an exposure/intervention group with a control group) and a measure of association between exposure and outcome | Written OR (e.g., OR 0.5, OR 1.34); reported a lot in health research | In the lecture's example paper the ORs were calculated using logistic regression

Odds – definition

Odds = probability of an event / probability of a non-event | Lecture example: the odds of rolling a 1 on a die = 1/5 = 20%

Interpreting an odds ratio: OR = 1, OR > 1, OR < 1

OR = 1: exposure does not affect the odds of the outcome | OR > 1: exposure indicates higher odds of the outcome | OR < 1: exposure indicates lower odds of the outcome

Interpreting ORs – dietary consumption example (parental work status and children's diet)

OR = 0.49: 2–3 year olds with part-time employed fathers are less likely to eat fruit and vegetables than children of full-time employed fathers | OR = 1.57: 4–5 year olds with part-time employed mothers are more likely to eat breakfast than children of full-time employed mothers

Is an odds ratio statistically significant? – using the p-value

Interpret p as normal: p < .05 → the OR is statistically significant | OR = 1.99, p = .017 → significant, higher odds | OR = .49, p = .002 → significant, lower odds | OR = 1.04, p = .689 → non-significant (even though > 1) | OR = .87, p = .347 → non-significant (even though < 1)

Is an odds ratio statistically significant? – using the 95% confidence interval

OR = 1 means no effect, so if the 95% CI includes 1 the OR is not significant | OR = 1.99, 95% CI [1.21, 2.30] → excludes 1 → significant, higher odds | OR = 1.24, 95% CI [.87, 1.56] → includes 1 → not significant

What does the 95% CI around an odds ratio tell you about precision?

It estimates how accurately we can predict the OR | Large CI = low precision | Small CI = higher precision

How does the confidence-interval rule differ between a t-test mean difference and an odds ratio?

For a t-test, the CI is around the mean difference; if it includes 0, there is no significant difference between means. For an odds ratio, the CI is around the OR; if it includes 1, there is no significant effect of exposure on the outcome.

Where do you select the nonparametric alternatives to the t-test in jamovi?

Independent Samples T-Test panel → Tests → Mann-Whitney U (independent groups) | Paired Samples T-Test panel → Tests → Wilcoxon rank (paired data)