ST411 Weeks 3-5: Models for binary response variables, General theory of GLMs, estimation, and inference

What kind of distribution is used for a binary response variable, and how is it defined?

Binary response variable Y_i uses the Bernoulli distribution

Y_i \sim \text{Bernoulli}(\pi_i)

-

\pi_i is the probability of outcome = 1 (success)

\pi_i = P(Y_i = 1)

What is the mean of a Bernoulli-distributed response variable Y_i?

Mean: E(Y_i) = \mu_i = \pi_i

-

\pi_i is the probability of success (i.e. Y_i=1)

What is the variance of a Bernoulli-distributed response variable Y_i?

Variance: \text{var}(Y_i) = \pi_i(1 - \pi_i)  

Why shouldn’t we use linear regression for binary Y?

Normality assumption is inappropriate for binary Y_i

Assumes constant variance \text{var}(Y_i) = \sigma^2

But for Bernoulli, var. depends on variable we’re modelling: \text{var}(Y_i) = \pi_i(1 - \pi_i)

Fitted probabilities \hat\pi_i can be outside [0,1] (impossible)

What is a link function in binary response models generally?

A link function g(\cdot) connects the mean \pi_i of the response variable to the linear predictor.

Model: g(\pi_i) = x_i'\beta

Ensures fitted values \pi_i = g^{-1}(x_i'\beta) stay in [0,1]

What is the logit link function (forward)?

g(\pi_i) = \log\left(\frac{\pi_i}{1 - \pi_i}\right) = x’_i \beta

-

Also written as \text{logit}(\pi_i)

\pi_i / (1 - \pi_i) is the odds of success

This transforms probabilities in (0,1) to the whole real line (-\infty , \infty)

What is the inverse of the logit link function (reverse) and what does it guarantee?

Inverse link: \pi_i = \frac{\exp(x_i'\beta)}{1 + \exp(x_i'\beta)}

-

Takes the linear combination xi’B and maps it back to probabilities pi. This is usefull when we generated predicted (fitted) probabilities.

Always gives values between 0 and 1

Keeps \pi_i within valid probability range

Binary logit model overview

• NOT written in mean+residual form

• No separate conditional variance parameter \sigma² , and no assumptions about it are needed

What are the odds that Y=1 in a logit model?

\frac{\pi}{1-\pi}

How do we interpret a coefficient \beta_j in a logistic regression model?

If x_j increases by a units

Then odds of Y = 1 are multiplied by \exp(a\beta_j)

This is the odds ratio comparing two values of x_j

How do we express the percent change in odds when x_j increases by a units?

Percent change in odds: (\exp(a\beta_j) - 1) \times 100\%

If \exp(a\beta_j) > 1, odds increase

If \exp(a\beta_j) < 1, odds decrease

In a logistic regression, what happens if we re-code Y=0 as in favour and Y=1 as against?

It simply reverses the signs of every coefficient. The fit and interpretation remain the same.

What are the odds a female has Y=1 (is in favour)?

\text{odds}(female) = \pi / (1- \pi) = 0.475/ (1-0.475) = 0.91

What are the odds a male has Y=1?

\text{odds}(male) = \pi / (1- \pi) = 0.537/ (1-0.537) = 1.1598

What is the odds ratio for Y=1 (being in favour for basic income) for men vs. women?

The odds ratio is calculated as \frac{odds(male)}{odds(female)} = \frac{1.1598}{0.91} \approx 1.276. It indicates how much more likely men are to be in favor compared to women. This means that the odds of men supporting UBI is are approximately 27.6% higher than female.

Write out the model for this regression and interpret this coefficient for the UBI example

\log\left(\frac{\pi_i}{1 - \pi_i}\right) = -0.1001 + 0.2476 \cdot \text{male}_i

Men have e^{0.2476} = exp(0.2476) = 1.28 times the odds of supporting basic income compared to women.

Men have 28% higher odds of supporting basic income than women.

Odds of female can be calculated from \beta_0 .

Interpret the coefficient on leftright

leftright is a continuous variable indicating political standing, so be careful.

The negative coefficient indicates that as we increase unit leftright, the odds are reduced.

Controlling for explanatory variables, increasing leftright by one unit, while holding all other variable constant, is associated with a odds in favour being multiplied by e^-0.11 =0.89 .

Controlling for other explanatory variables, a one-point increase in leftright is associated with a 11% reduction in the odds of supporting basic income.

Interpret the coefficient on educupper secondary

Having an upper secondary education (relative to lower secondary) multiplies the odds of supporting basic income by e^{-0.186} = 0.83 , holding all other explanatory variables constant.

This means that individuals with upper secondary education have 17% lower odds of supporting basic income compared to those with lower secondary education holding all other explanatory variables constant.

Interpret the coefficient on age

Controlling for gender, education, and political standing, a one-year increase in age is associated with a e^{-0.014} = 0.986 multiplies the odds of supporting basic income by 0.986.

Controlling for gender, education, and political standing, a one-year increase in age is associated with a 1.4% decrease in the odds of supporting basic income.