Week 1

Benefits of Regression

Regression analysis is a set of statistical methods used for the estimation of relationships between a dependent variable and one or more independent variables. It can be utilized to assess the strength of the relationship between variables and for modelling the future relationship between them.

Flexibility

• in terms of hypothesis and data analysis

• regression allows you test hypothesis using the null hypothesis significance testing framework, as each model will give you associated p values

• you can model both continuous and categorical predictors at the same time

  • in an ANOVA you have to transform a continuous variable into a categorical variable for analysis, but you don’t in regression

• regression can model secondary variables with ease, e.g. extraneous, third, or nuisance. you can out them into the model at the same time as primary predictors.

• model standardised data that allows you to compare the strength of relationships on the outcome variable across other predictors

  • other models require you to separately make effect sizes to do this

• no need for post-hoc testing

Statistical power

• regression retains a higher statistical power to detect effects than ANOVA

  • ANOVA reduces all participant observations in a level of a factor down to one value (the mean).

  • It also reduces standard deviation

  • Regression can do this but it analyses the data at the level of the individual, taking all the participant observations into account

Predictions

• regression analysis can generate predictions

• once the model is built a regression equation or a computer package to generate predictions for data values that aren’t in the data set.

  • can help generate hypothesis for future research

Extensions

• regression can extend into other types of data form e.g. binary, longitudinal or ranked data

Limitations of Regression

More variables

• potential to mix categorical type variables with continuous variables

more data preparation and visualisation

the intercept term

• intercept term is a constant → we usually ignore it and do not interpret it but it is still needed

• intercept term is never explained in a regression model

• when you have lots of predictor variables the intercept term represents little parts of every predictor variable and the other predictor coefficients are other parts of every predictor variable

• if you understand the intercept term it becomes easier to interpret your other models

Reference levels

Understanding how variables work together or making predictions

Correlation: Pearson’s r

• this is a relationship between two continuous variables

• has two properties of the relationship:

  • a strength

  • a direction

• standardised variables

  • can only have a value between -1 and 1

• no relationship = independent/uncorrelated