Investigating hypotheses

What is the purpose of analysis

Compare outcomes between groups

Assess agreement between groups

Assess associations between variables

Compare matched vs paired data

**Paired data**

* 2 sets of related/non-independent data
* Eg – siblings, repeated from same person

**Matched data**

* Extension of paired data for more than two groups

When are parametric methods used?

* Good to detect genuine differences
* Can only be used for continuous normal outcome

When are non-parametric methods used?

* Less powerful to detect genuine differences
* Can be used for any numerical outcome - non-normal continuous/discrete outcomes

What are non-parametric methods based on?

Ranks instead of actual data values

Parametric tests used for comparing numerical outcomes between the following groups

* Comparing 2 unpaired groups
* Comparing 2 paired groups
* Comparing 3+ unmatched groups
* Comparing 3+ matched groups

Unpaired t test

Paired t test

One-way ANOVA

Repeated-measures ANOVA

Non-parametric tests used for comparing numerical outcomes between the following groups

* Comparing 2 unpaired groups
* Comparing 2 paired groups
* Comparing 3+ unmatched groups
* Comparing 3+ matched groups

Mann-Whitney U/Wilcoxon rank sum test

Sign test/Wilcoxon signed rank test

Kruskal-Wallis test

Friedman test

Non-paramtetric methods used for comparing categorical outcomes between the following groups

* Comparing 2 unpaired groups
* Comparing 2 paired groups
* Comparing 3+ unmatched groups
* Comparing 3+ matched groups

X2 test

McNemar’s test

X2 or ordinal X2 test

Cochrane’s Q test

When do you need to assess agreement

when you are comparing outcomes measured by multiple assessors/clinical methods

Aim is always to obtain the same value from different sets of outcome measurements

How could measurements be strongly associated but have low agreement?

If 2 assessors collect data, one may consistently score higher than the other

inter-observer variability

= accuracy/assessment of agreement for different assessors/methods

intra-observer variability

assessment of agreement for same assessor/method

What is Bland-Altman used to assess

Limits of agreement between numerical outcomes

How do you assess limits of agreement?

1. Calculate the difference between 2 sets of measurements
2. Calculate the mean of 2 measurements
3. Plot
4. Limits of agreement = Mean difference +/- 2 SD of the difference


1. Give range of agreement to judge whether differences are clinically important or not

Method to assess agreement between categorical outcomes

Method to assess agreement between numerical outcomes

Kappa/weighted kappa statistics

Limits of agreement (Bland-Altman)

Use of Kappa statistics

Based on comparing observed proportion of agreement with the proportion of agreement that would be expected to occur by chance – binary/nominal outcomes

For what type of outcome is weighted kappa more appropriate?

ordinal outcomes

Includes partial misclassifications

What type of agreement does 1 suggest in kappa statistics?

perfect agreement

What type of agreement does 0 suggest in kappa statistics?

no agreement

what type of data should kappa values be presented with?

SEs or Cis rather than p values