Q18: F stats

Grand mean

The average of all observations from every group

Group mean

The average of the observations in one group

Observation

One measured value from a person animal or other item in a study

Deviation

The difference between a value and another value such as a mean

Model deviation

The difference between a group mean and the grand mean

Residual

The difference between one observation and its own group mean

Total deviation

The difference between one observation and the grand mean

Fitted value

The group mean predicted by the model for an observation

Sum of squares (SS)

A total found by squaring deviations and adding them together

Total sum of squares (SS_T)

The sum of squared differences between every observation and the grand mean

Model sum of squares (SS_M)

The sum of squared differences between each fitted group mean and the grand mean

Error sum of squares (SS_E)

The sum of squared differences between observations and their own group means

SS total formula

SS_T = sum of (observation minus grand mean) squared

SS model formula

SS_M = sum of (fitted value minus grand mean) squared

SS error formula

SS_E = sum of (observation minus fitted value) squared

Sum of squares rule

SS_T = SS_M + SS_E which means total variation equals model variation plus error variation

Variation

How much the values differ from one another

Between-group variation

How much the group means differ from the grand mean

Within-group variation

How much observations differ from their own group mean

Explained variation

The part of the total variation accounted for by differences between group means in the model

Unexplained variation

The part of the total variation left within the groups

Degrees of freedom (df)

The number of independent pieces of information used in a calculation

Number of observations (n)

The total number of measured values in the data

Number of groups (k)

The total number of groups being compared

Model degrees of freedom

The number of groups minus one or k - 1

Error degrees of freedom

The number of observations minus the number of groups or n - k

Total degrees of freedom

The number of observations minus one or n - 1

Mean square (MS)

A sum of squares divided by its degrees of freedom

Model mean square (MS_M)

The model sum of squares divided by the model degrees of freedom

Error mean square (MS_E)

The error sum of squares divided by the error degrees of freedom

Model mean square formula

MS_M = SS_M / (k - 1)

Error mean square formula

MS_E = SS_E / (n - k)

F-statistic

A number that compares between-group variation with within-group variation using mean squares

F formula

F = MS_M / MS_E

Large F-statistic

An F-value showing that between-group variation is large compared with within-group variation

Small F-statistic

An F-value showing that between-group variation is small compared with within-group variation

Can F be negative

No because sums of squares and mean squares cannot be negative

Null hypothesis

The starting claim that the group means are equal

Alternative hypothesis for two groups

The claim that the two group means are different

P-value

The chance of getting a test result at least as extreme as the observed result if the null hypothesis and test assumptions are true

Statistical significance

A result that meets the chosen cutoff for evidence against the null hypothesis

F-distribution

The distribution used to judge an F-statistic when testing a model

Two-sample t-test

A test that compares the means of two groups

ANOVA

A method that compares group means by examining between-group and within-group variation

Relationship between F and t

For two groups using the standard equal-variance methods F = t squared

Why square t to get F

Squaring removes the negative sign and gives the corresponding nonnegative F-statistic

R-squared (R²)

The share of total variation explained by the model

R-squared formula

R² = SS_M / SS_T

R-squared of 0

The model explains none of the observed total variation

R-squared of 0.60

The model explains 60 percent of the observed total variation

R-squared of 1

The model explains all of the observed total variation

What happens to F when group means move farther apart

F generally increases if within-group variation stays the same

What happens to F when within-group variation increases

F generally decreases if the group means stay the same

What happens to R² when within-group variation increases

R² decreases if the model sum of squares stays the same

What happens when group means become more similar

Model variation and usually F and R² decrease if within-group variation stays similar

Balanced groups

Groups with similar numbers of observations

Why group balance matters

Group sizes can affect the stability of the estimated group means and R²

Figure 1 Panel A

Shows model deviations from group means to the grand mean

Figure 1 Panel B

Shows residuals from individual observations to their own group means

Figure 1 Panel C

Shows total deviations from individual observations to the grand mean

Main lesson of Figure 1

Total deviation splits into model deviation and residual deviation and the sums of squares satisfy SS_T = SS_M + SS_E

Figure 2

Shows how the average R² changes as the number of white individuals in the sample changes

What does the dashed line in Figure 2 suggest

The average R² tends to level off as the group composition reaches a more stable range

Does a high R² prove cause and effect

No it only describes the share of variation explained by the model

Does a large F automatically mean a significant result

No you must use the F-distribution and the correct degrees of freedom to find the p-value or compare with a cutoff

What happens to F when model variation is much larger than error variation

F becomes large

How to check sums of squares

Add SS_M and SS_E and check that their sum equals SS_T

What does augment do in R

It adds fitted values and residuals to the model data

What does .fitted mean in R

The value predicted by the model for each observation

What does .resid mean in R

The difference between the observed value and the fitted value

What does anova(fit) show in R

The sums of squares degrees of freedom mean squares and F-statistic for the model

What does r.squared show in R

The share of total variation explained by the fitted model

What happens when there are more than two groups

ANOVA can compare all the group means using the same general approac