Friedman Test and Categorical Data Analysis

Friedman Test

• Non-parametric equivalent of the One-way repeated measures ANOVA and an extension of the Wilcoxon signed-rank test.

• Used for comparing 3+ dependent groups.

• Requires 3+ observations/measurements for each "participant."

Main Assumptions of Repeated Measures ANOVA

• Normality of residuals

• Sphericity

• If one or more of these assumptions fail, consider the Fredman test.

• Used for continuous or ordinal data.

Hypotheses

• H_0: All groups belong to the same distribution.

• H_a: At least one group belongs to a different distribution.

• If the distributions of the groups are of the same shape, the hypotheses are:

  • H_0: The medians of all groups are equal.

  • H_a: At least one group has a median not equal to the others.

Method

1. State H_0

2. By construction, each group will have the same number of entries, denoted as n, and let k be the number of groups.

3. Rank the data for each participant across groups from 1 to k (Potential ranks: 1, 2, …, k).

4. If we have repeated values across groups, assign a tied rank, which is the mean of the potential ranks of these data (Actual ranks).

5. Sum the ranks for each group, denoted by R_i for the i^{th} group.

Test Statistic

• The test statistic is given by:
  F = [\frac{12}{nk(k+1)} \sum{i=1}^{k} Ri^2] - 3n(k+1)

• If we have tied ranks, we use the corrected test statistic:
  F{corrected} = \frac{(k-1) [\sum{i=1}^{k} Ri^2 - CF]}{\sum{i=1}^{n} \sum{j=1}^{k} r{ij}^2 - CF}
  where CF = \frac{n k (k+1)^2}{4} and r{ij} is the rank corresponding to group i in participant j.

• For k = 3 and k = 4, and n = 2, 3, 4, we evaluate the test statistic by using tables.

• If F/F{corrected} > F{crit}, we reject H_0. Otherwise, we use \chi^2 tables with k-1 degrees of freedom.

• If F/F{corrected} > \chi^2{crit}, we reject H_0.

Post Hoc Tests

• Wilcoxon signed-rank tests on all pairs. However, this tends to inflate the Type I error rate.

• To control this, we use the following Bonferroni correction.

• Nemenyi post hoc tests.

Categorical Data Analysis

• Categorical data take nominal values, i.e., no intrinsic ordering (e.g., eye color).

• We address cases where both predictor and outcome variables are categorical (e.g., association between personality and color preference).

• We cannot compare means with categorical data.

• Consider frequencies instead.

Assumptions for Categorical Variables

• Independence: Each person, item, etc., only contributes to one cell in the frequency table.

• Expected Frequencies:

  • For \chi^2 goodness-of-fit test and 2 x 2 tables, no expected frequencies should be < 5.

  • In longer tables, all expected frequencies > 1 and no more than 20% of expected frequencies < 5.

The \chi^2 Tests

• These can be used for goodness-of-fit and to test for association between categorical variables.

The Chi-Square (\chi^2) Distribution

• Let Z1, …, Zk \sim N(0, 1), then
  S = \sum{i=1}^{k} Zi^2 \sim \chi^2_v

• S has a probability density function (pdf):
  P_v(x) = \begin{cases} \frac{1}{2^{v/2} \Gamma(v/2)} x^{v/2 - 1} e^{-x/2}, & x > 0 \ 0, & \text{otherwise} \end{cases}
  where \Gamma denotes the Gamma function.

• For n \in \mathbb{N}, \Gamma(n) = (n-1)!

• For z \in \mathbb{C}, \Re(z) > 0,
  \Gamma(z) := \int_{0}^{\infty} x^{z-1} e^{-x} dx

Properties

• \chi^2 distributions have a positive skew.

• As v \rightarrow \infty, \chi^2 becomes symmetrical.

• Let X \sim \chi^2_v, then
  E(X) = v
  Var(X) = 2v
  \frac{X - v}{\sqrt{2v}} \sim N(0, 1)

Calculations

• If X \sim \chi^2v, then E(X) = \int x Pv(x) dx = v
  Var(X) = E(X^2) - [E(X)]^2 = 2v
  E(X^2) = \int x^2 P_v(x) dx = (v+2)v

• If \lbrace Xi \rbrace{i=1}^{N} is a sequence of independent \chi^2 random variables with d.f. v1, v2, …, vN respectively, then \sum{i=1}^{N} Xi \sim \chi^2{\sum{i=1}^{N} vi}

\chi^2 Goodness-of-Fit Test

• Used to test if categorical data are as expected, or if it fits a certain distribution.

Example

• Exam Mark classification: A, B, C, D, E

• Exam Board: expectation of frequencies in each class.

• Exam Results: test to compare with expected frequencies to determine any potential issues.

Test Statistic

• The test statistic is:
  X^2 = \sum{i=1}^{K} \frac{(Oi - Ei)^2}{Ei}
  where K is the number of categories.

• O_i are the observed frequencies.

• E_i are the expected frequencies.

Evaluation

• Evaluate X^2 using \chi^2 tables with K-1 degrees of freedom.

• Hypotheses:

  • H_0: X^2 = 0 (no difference between observed and expected frequencies).

  • H_1: X^2 \neq 0 (a significant difference between the observed and expected frequencies).

• If X^2 > \chi^2{crit}, we reject H0.