Multiple Regression Analysis

Multiple regression is similar to simple regression but involves multiple independent variables (X's) to predict an outcome (Y).
Formula: Predicted Y = $β1X1 + β2X2 + β3X3 + … + A$ (where A is the constant or intercept).
It assesses how well a group of variables predicts the outcome variable.
Topics covered include model fit, assumptions, strength of association, inference, and variable selection.

Predictors (independent variables or X's) should be theory-driven or based on empirical data.
Avoid randomly combining variables without a theoretical or empirical basis.
Example: Predicting life satisfaction (Y) based on income (X1), partnership status (X2), and health (X3).

Venn diagrams illustrate the variance explained by each predictor and the overlap between predictors.
Some predictors uniquely explain the outcome, while others share variance.
Example: Income might uniquely predict life satisfaction, while partnership status and health may have overlapping variance.

R-squared represents the proportion of variance in the dependent variable (Y) explained by the independent variables (X's).
Calculated as: $R^2 = \frac{SS{explained}}{SS{total}}$ (where SS is the sum of squares).
It indicates how much of the variance is explained by the model versus how much is left over (error).
Example: An R-squared of 0.25 means 25% of the variance in Y is explained by the X's.

Beta values indicate the average unit change in Y for each unit change in X, holding other X's constant.
Standardized beta values allow comparison of the strength of association between predictors.
Standardized values of X and Y help determine the strongest predictor by comparing beta values.
Unstandardized values can be difficult to interpret because different variables have different scales.
It's possible to determine which predictors significantly predict the outcome and rank them by importance.

Independence: Observations are independent of each other.
Linearity: The relationship between variables is linear.
Bivariate Normality: Variables are normally distributed.
Homoscedasticity: The variance of errors is constant across all levels of the independent variables.
Non-multicollinearity: Predictors (X's) are not highly correlated with each other.
Statistical tests can assess multicollinearity.

The F-test is used to determine the significance of the overall model.
Formula: $F = \frac{R^2 / k}{(1 - R^2) / (n - k - 1)}$ (where R-squared is the proportion of variance explained, k is the number of predictors, and n is the sample size).
Less predictors explaining a large proportion of variance result in a stronger finding.
Larger sample sizes increase the likelihood of significance.
The formula considers the proportion of variance explained, the number of predictors, and the sample size.
Less predictors and larger sample size increase the likelihood of a significant finding.