Multiple Regression

Lecture Overview

  • This lecture covers the assumptions of multiple regression and the importance of making inferences about population parameters rather than just interpreting sample data.

  • Emphasizes the necessity of meeting parametric assumptions to conduct valid analyses.

Introduction to P-Values and Regression Output

  • Initial focus on understanding the output of multiple regression, specifically the significance of p-values.

  • Overview of regression output tables:
      - Overall Model Significance Table: Indicates whether the multiple R explains a significant amount of variance in the dependent variable.   - Predictor Significance Table: Shows the significance of each predictor variable.

  • Example:   - Non-significant predictors: private sale and dealership.   - Significant predictor: kilometers (p < 0.05).

Confidence Intervals

  • Understanding confidence intervals:
      - If both upper and lower bounds share the same sign (both negative or both positive), the interval is significant.
      - If one is positive and one is negative, the interval is not significant.

Assumptions in Multiple Regression

  • Primary assumptions to be checked include:   - Multicollinearity
      - Normality
      - Linearity
      - Homoscedasticity
      - Independence

Multicollinearity

  • Definition: Multicollinearity occurs when two or more predictor variables are highly correlated with one another.
      - This correlation can make the regression coefficients unstable and lead to misleading results.

  • Importance of avoiding high correlation among predictors:   - Redundant predictors do not contribute unique information to the model.

  • Diagnosis of multicollinearity:   - Conceptual understanding: Assessing which predictors might be too highly correlated.   - Practical testing:     - Run a multiple regression with one predictor serving as the dependent variable.     - Evaluate the correlation with remaining predictors.

  • Example Analysis:   - If kilometers is highly correlated with other predictors, leading to inflated R-squared values,   - Example showing adjusted R-squared of 0.95, indicating problematic multicollinearity.

  • Tolerance and Variance Inflation Factor (VIF):   - Tolerance values must be greater than 0.1 to avoid multicollinearity issues.   - VIF values over 10 indicate multicollinearity problems.

  • Solutions for multicollinearity:   - Combine highly correlated variables by averaging scores.   - Delete one of the predictors based on psychometric properties or theoretical relevance.

Residual Assumptions

  • Residuals need to meet the following assumptions:   - Normality
      - Linearity
      - Homoscedasticity
      - Independence

Normality of Residuals

  • Histogram and PP plot analysis:   - Histogram to visually assess the normality of residuals.   - PP plot to check fit along the diagonal line for normal distribution.

Linearity of Residuals

  • Visual checks for linearity using scatter plots:   - A horseshoe pattern indicates non-linearity, while a sufficiently straight pattern shows compliance with linearity.

Homoscedasticity of Residuals

  • Definition: Homoscedasticity means that residuals have constant variance across all levels of the independent variable(s).

  • Identifying heteroscedasticity:   - Fan-shaped patterns in residual plots indicate problems.

  • Example of problematic heteroscedasticity provided.

Addressing Non-Normality and Heteroscedasticity

  • Impact of violation on p-values:   - Non-normality or heteroscedasticity affects p-values, not the size of beta weights or R-squared values.

  • Solutions include:   - Transforming independent and dependent variables.   - Utilizing bootstrapping methods for statistical analysis.

  • Handling pronounced non-linearity:   - Square or cube the predictors for linear relationships.

Independence of Residuals

  • Considerations of independence include:   - Clustering: Sampling from distinct groups (e.g., classrooms, different sales types) may bias results.     - Include potential clustering variables in the regression model.   - Order Effects: Systematic changes over time can introduce non-independence (e.g., learning effects).

  • Use of Durbin-Watson statistic:   - Computes correlation between residuals to check independence.   - Values between 1.5 and 2.5 are acceptable (example shows value 1.8).

Cross-Validation of the Model

  • Importance of cross-validation:   - Ensures robustness of model results when applied to different samples from the population.

  • Reporting Adjusted R-squared:   - Provided by statistical software (e.g., SPSS) as a measure of model fit.

  • Stein's formula mentioned as alternative but involves additional calculations not required for this unit.

  • Suggestion to split data for dual regression runs when feasible to assess consistency of results.

Conclusion

  • Summarizes the critical aspects of multiple regression assumptions.

  • Encouragement to consider these assumptions in future analysis.

  • Invitation to the next lecture for further exploration of regression topics.