W13 - Generalised linear models + logistic regression

general linear models

outcome variable is linear combination of predictors + error

• assumed to be continuous/interval and normally distributed

model directly predicts values of the outcomes itself

models are flexible

• in number and type of predictors they include

• but not in the kinds of outcomes they can model

• once outcome is no longer continuous and unbounded, assumptions break down

  • a restrictive model

eg. linear regression, ANOVA and mediations

generalised linear models (GLMs)

use when outcome variable is not normally distributed or measured on a continuous/interval scale

not a single statistical model

• a framework of various statistical analyses

still retain the familiar regression structure for combining predictors

allows us to make the changes of different probability distributions and link functions

• while keeping the regression-style combination of predictors

key differences between general and generalised linear models

type of outcome variable being modelled

• general = continuous/interval data following normal distribution

• generalised = other data not following normal distribution

  • eg. binary, counts, skewed

GLMs extend the general linear models framework by:

relaxing the outcome-related assumptions

allowing outcome to follow different distributions than a normal one, that better reflect the data

• eg. binary distribution for yes/no outcomes

GLM examples

binary logistic regression

• 2 options

• eg. ‘are you happy’ yes or no

multinomial logistic regression

poisson regression

• eg. ‘count how many happy thoughts you’ve had today’

negative binomial regression

ordinal logistic regression

• eg. ‘rank how happy you are on this ordered list of options’

binary logistic regression

most common GLM

used when outcome variable is binary

• has 2 possible values

• coded as 0 and 1

• eg. yes/no questions

models the relationship between 1/+ predictors and an outcome

• predictors can be continuous, categorical or a mixture

instead of predicting the outcome directly, it models the probability that the outcome occurs

useful when research is concerned with likelihood of an event

• rather than the size of a continuous outcome

the coefficients are log-odds, rather than probabilities

the Bernoulli distribution

describes a binary outcome

the distribution is defined by a single parameter

• represents the probability that the outcome takes the value 1

  • eg. if the probability of reporting anxiety is 0.30, then each observation has a 30% chance of being coded as 1, and a 70% chance of being coded as 0 (100%-30%)

distribution is not symmetric and doesn’t describe variability around a mean in the usual sense

• reason why linear regression is inappropriate for binary outcomes

suitable for modelling binary outcome variables as it only has 2 outcomes - dichotomous

logistic regression axis’

x-axis

• represents the 2 possible observations (0/1)

y-axis

• shows the probability of each outcome

  • must equal 100% for both values

• log-odds, not probabilities

logistic regression assumptions

different to linear regression, as the outcome variable is binary not continuous

• observations assumed to be independent

  • each data entry should represent a separate observation

  • violations affect SEs and inference, rather than the estimated relationships themselves

• relationship between predictors and log-odds of outcome is assumed to be linear

  • assumes linearity on log-odds scale, not between predictors and probabilities

  • violations make model fit poorly, even if predictors appear related to the outcome

• should be no perfect multicollinearity among predictors

  • they shouldn’t be exact linear combinations of other predictors

  • high levels make it difficult to estimate and interpret individual effects reliably - just like linear regressions

log-odds

logistic regression coefficients

odds of an event = the ratio of the probability that the event occurs, to the probability that it doesn’t

• log-odds are the logarithm of these odds

they are not bounded

• can take any value from -infinity to +infinity

normal probabilities are constrained between 0 and 1 (0-100%)

• creates problems for regression models, where predictors are combined linearly and could take any value

  • log regression resolves this by transforming probabilities into odds, then log-odds

aim to change these log-odds into odds/probabilities to allow for easier interpretation

difference between log-odds, odds and probabilities

they all measure the likelihood of events happening, but on different scales

probabilities - successful outcomes/total trials

• bounded between 0 and 1 = unsuitable for linear modeling

odds - successful outcomes/failed outcomes

• bounded from 0 to infinity

log-odds - logarithm of these odds

• maps probabilities to a range from -infinity to +infinity

• perfect for linear regression models

poisson regression

when outcome variable is count data

number of symptoms reported, events occurring in fixed time frame, times a behaviour is observed

• eg. ‘count how many happy thoughts you’ve had today’

assumes the outcome follows a Poisson distribution

negative binomial regression

when outcome variable is count data

more flexible than Poisson regression, when the data shows greater variability than that model allows

• common in real-world psychological and health data

multinomial logistic regression

extends logistic regression to outcomes with 2+ categories

• where they are nominal and unordered

  • eg. choice of treatment type

useful as a natural extension

ordinal logistic regression

when outcome variable has ordered categories

• eg. likert-type response

• explicitly account for the ordering of categories, rather than treating them as purely categorical

more advanced form of GLM

odds ratio (OR)

OR < 1

• reduced probability

OR = 1

• equal odd of either outcome

OR > 1

• greater chance of outcome