PSCH 443 Multiple Regression Overview and Example

Overview of Multiple Regression

• Definition: Multiple regression aims to explain the variation in an outcome variable (Y) using multiple predictor variables (X).

• Goal: Similar to bivariate regression, it seeks to explain why some individuals score high or low on the Y variable.

• Overlap Consideration: It is essential to account for the unique contributions of each predictor as well as the overlap between predictors themselves concerning their influence on the outcome variable.

Model Parameters and Techniques

• Estimate Model Parameters:

  • Use least squares techniques for estimating model parameters.

  • Include an intercept, multiple B parameters, and regression weights for each predictor.

• Evaluate Model Fit:

  • Assess overall model fit using R-squared.

  • Examine individual predictors' contributions to ascertain if they add meaningful value to the model.

Structure of the Multiple Regression Equation

• Generic Structure: Similar to a simple regression equation, including:

  • A parameter: Intercept (value of Y when all X's are zero).

  • B weights: Slope parameters for each predictor variable.

  • Error term: Accounts for variability not explained by the predictors.

Example of Multiple Regression Analysis

Context

• Scenario: Assess whether GRE scores can predict graduate school performance.

• Dataset Details:

  • 36 participants with variables including undergraduate GPA, GRE scores, and years to complete a master's degree.

• Predictors:

  • GRE scores (verbal sub-score)

  • Undergraduate GPA

• Outcome: Years taken to complete a master's program.

Initial Steps

• Examine Bivariate Relationships:

  • Look at the correlation matrix between variables to identify expected associations (e.g., higher GPA correlates with higher GRE scores).

  • Check for potential issues such as unexpected correlation directions or strong correlations between predictor variables (multicollinearity).

Model Summary Statistics

• Multiple Correlation (R): Correlation of all predictor variables with the outcome Y variable.

• R-squared Value:

  • Represents the proportion of variance explained by predictors; in this example, about 26% of the variance in completion years is explained.

• Adjusted R-squared:

  • Adjusts R-squared based on sample size and number of predictors; relevant when considering generalizability of the results.

• Standard Error of Estimate: Reflects average prediction error, relevant for model accuracy.

ANOVA and Degrees of Freedom

• ANOVA Overview: Ratio of effect sums of squares to error sums of squares to test the overall significance of the model.

• Degrees of Freedom: Determines total degrees of freedom based on the number of participants.

  • Total DF: Total participants - 1 (e.g., 35 for 36 participants).

  • Regression DF: Coefficients estimated - 1 (e.g., 2 predictors + intercept = 3 - 1 = 2).

Coefficient Interpretation

• Constant (Intercept): Found in the table as unstandardized coefficients.

• B Weights:

  • Indicate the change in Y for a one-unit change in X.

  • Example: Holding GPA constant, each GRE point results in a 0.002 years reduction in completion time; a full GPA grade point increase results in a 0.634 years reduction.

• Standardized Coefficients (Beta Weights): Reflect changes in Y in standard deviation units, allowing comparison of predictor effects.

  • GRE: A standard deviation increase predicts a 0.201 standard deviation reduction in years to completion.

  • GPA: A standard deviation increase predicts a 0.393 standard deviation reduction in years to completion.

Statistical Significance of Coefficients

• T Statistics: Evaluate significance of B parameters (unstandardized coefficients).

  • GRE scores significance: P = 0.233 (not significant).

  • Undergraduate GPA significance: P = 0.023 (significant).

• Practical Importance of Effects: Consider the size of coefficients in relation to real-world applications beyond p-values; significant testing alone does not confirm practical importance.

Assumptions and Next Steps

• Importance of Assumptions: Results interpretation depends on meeting regression assumptions, types of regression used, and the overall purpose of the analysis.

• Critical Evaluation: Emphasize understanding beyond statistical significance for practical application and insights.