1 - multiple regression

what is regression used for?

-used to predict variance in an outcome variable (DV) from one or more predictor variables (IVs)

-eg predicting work absence from office culture, health and safety + resilience

what can correlational research establish + what are its limitations?

-establishes whether variables are associated, but it cannot establish causation because the data are observational

-even when one variable changes before another, a third variable could explain the association

why is correlational research important despite its limitiations?

-allows us to investigate things that cannot ethically or feasibly be studied experimentally, such as drug addiction, disease, chronic stress + age

-also useful for questionnaires, surveys, secondary data analysis, exploring big data + making predictions

how do we describe direction + strength of a relationship?

• Direction: positive or negative.

• Strength: how closely observations cluster around the line of best fit

• strong association means knowing X allows us to estimate Y more accurately

• weak association means there is more variation around the line, making predictions less accurate

what does regression aim to tell us?

-whether the model is a good fit

-whether significant relationships exist between predictors + the outcome +the direction of those relationships

-these results can then be used to make predictions

what is the regression equation + how is line of best fit selected?

Y = bx + a

Y = predicted outcome.

x = predictor value.

b = slope of the line.

a = intercept

line of best fit is selected by minimising the sum of squared prediction errors — the squared differences between observed and predicted values

what are SSR, SSE + SST?

• SSR (sum of squares regression): the variation explained by the regression model

• SSE (sum of squares error): the variation not explained by the model

• SST (sum of squares total): the total variation

SST = SSR + SSE

better-fitting model generally explains more variation + leaves less unexplained variation

what is R2 + how is it calculated?

-proportion of total variation in the outcome explained by the regression model

R2 = SSR divided by SST = SSR divided by SSR + SSE

-ranges from 0-1, multiply by 100 to express as %

what is adjusted R2 + why is it useful?

-measures model fit while adjusting for the number of predictors. -usually lower than \(R^2\).

-adding predictors will increase or maintain \(R^2\), even if they contribute little useful information

-adjusted \(R^2\) accounts for this + can decrease when unhelpful predictors are added

how do we determine whether the overall regression model is statistically significant?

ANOVA is used to test whether the proportion of variance in the DV predicted by the IV(s) is statistically significant

-output provides an F-statistic, degrees of freedom + p-value

R2 + adjusted R2 describe model fit, while the ANOVA tests the model's statistical significance

Why isn't a significant overall regression model enough to understand individual predictors?

-overall model tells us whether the predictors collectively explain a significant amount of variance, but it does not tell us each predictor's individual contribution or direction of association

-we need to examine the individual regression coefficients

what does an unstandardised regression coefficient tell us?

-tells us how many units the DV changes for each one-unit increase in the IV

-B = 2.00 : the DV increases by 2 units

-B = -2.00: the DV decreases by 2 units

-B = .03 : the DV increases by .03 units

-B = -.01: the DV decreases by .01 units

what is the standard error (SE) + how is it used to calculate the t- statistic?

-SE indicates the uncertainty or precision of the estimated regression coefficient

-smaller SE generally indicates a more precise estimate

t = B divided by SE

-coefficient’s size relative to its SE determined t-statistic, which is used to calculate the p value

-generally, larger absolute t stat corresponds to smaller p value

what are standardised regression coefficients + why are they useful?

-beta expresses the association in standard deviation units.

-eg For example, beta=.50 means a one-standard-deviation increase in the IV is associated with a .50-standard-deviation increase in the DV, if beta=-.50, the DV decreases by .50 standard deviations

-beta allows us to compare the relative strength of associations between different predictors + the same DV.

-larger absolute beta indicates a stronger association. -standardised + unstandardised coefficients have the same p-value for the same predictor

what are the assumptions of simple + multiple regression?

-outcome is approximately normally distributed and continuous.

-observations are independent’

-predictors are interval/ratio or nominal with two categories (dichotomous)

-multiple regression requires checking for multicollinearity

what are the main benefits of multiple regression?

• examines several predictors of one outcome.

• estimates each predictor's effect while holding the other predictors constant.

• identifies each predictor's unique contribution to explaining the DV

what is multicollinearity + why is it a problem?

-occurs when independent variables in a regression model are highly correlated

-predictors provide overlapping rather than unique information. -this can make regression coefficients + standard errors unreliable + a model may explain a large amount of variance even when none of its individual predictors is statistically significant

how can you identify multicollinearity using a correlation matrix?

-look for high correlations between independent variables

-r >.80 as a rule of thumb

-an r = 1.00 correlation indicates perfect multicollinearity + may suggest a data issue

what is the tolerance statistic + how is it interpreted?

-indicates how much of a predictor's variance is not explained by the other predictors

-tolerance = 1 - R2

-high tolerance = low multicollinearity

-low tolerance = high multicollinearity

-values below .20/ .10 can indicate a problem

what is variance inflation factor (VIF) + what do its values mean?

-inverse of tolerance + indicates how much the standard error is inflated due to multicollinearity

VIF = 1 divided by tolerance

-VIF above 4 suggests multicollinearity

-VIF above 10 suggests substantial multicollinearity

what can researchers do if multicollinearity is a problem?

-increase the sample size to stabilise regression coefficients.

-remove redundant variables

-combine variables into a new variable if two or more are important + measure overlapping information

summary

adjusted R2: how much variance the model explains, accounting for the number of predictors

F-statistic + p-value: whether the overall model is statistically significant

B coefficients: how much + in what direction each predictor relates to the DV

p-values for individual predictors: whether each predictor's association is statistically significant

Multicollinearity checks: whether predictors overlap too strongly