Study Notes on Multiple Regression

Overview of the Unit

  • Covered Module One

  • Looked into correlations

  • Currently at Multiple Regression

Transition to Multiple Regression

  • Multiple regression is significantly foundational in research.

  • Personal preference for statistical methods:
      - If restricted to one method: Multiple Regression
      - In reality, choice would be Structural Equation Modeling (SEM)

  • Comparison between SEM and Multiple Regression:
      - SEM akin to riding a unicycle: more complex
      - Multiple Regression feels more accessible, like riding a bicycle

Lecture Objectives

  • Conceptual overview of multiple regression

  • Worked example related to car purchase using multiple regression

  • Step-by-step breakdown of research steps:
      - Power calculations
      - Outlier assessment
      - Initial assumption testing (moved to a separate lecture for clarity)

  • Interpretation of output

  • Summary of multiple regression in three slides

  • Brief insight into the mechanics of regression

Definition of Multiple Regression

  • Involves two or more independent variables (IVs) to predict a single dependent variable (DV).

  • Comparison to Simple Regression:
      - Simple Regression: only one IV
      - Multiple Regression: multiple IVs (continuous or categorical)

Independent & Dependent Variables

  • Predictors (IVs):
      - Primarily continuous, can also be dichotomous (binary)
      - Use of dummy coding to include categorical variables

  • Dependent Variable (DV):
      - Must be continuous or a scaled variable

Methods and Related Concepts

  • Current focus: Multiple Regression

  • Future topics: Assumptions of Regression, Hierarchical Regression, Moderation, and Mediation

Predicting Child IQ Example

  • DV: Child IQ at age 10 (continuous variable)

  • IVs:
      - Mother’s smoking
      - Months of breastfeeding

  • Transformation from correlation to multiple regression:
      - Involves multiple r instead of a single r
      - Squared multiple r gives percentage of variance explained in DV by IVs combined

Understanding the Output

  • Introduction of Beta Weights:
      - Each IV has a beta weight that indicates the relationship with the DV
      - Multiple r captures combined effect of beta weights

Nature of Multiple r

  • Properties of Multiple r:
      - Can only be positive (unlike r which can be both)
      - Reflects the cumulative effect of positive and negative beta weights

  • Limitations of Multiple r:
      - Potential bias in favor of positive association
      - Use of squared multiple r to determine explained variance

Application Example: Car Pricing

  • Objective: Determine how IVs (kilometers, year model, sale type) predict car price
      - Required analysis steps include:
        - Research question formulation
        - Statistical approach selection (Multiple Regression)
        - Hypothesis development
        - Power computation (post hoc analysis with pre-set alpha levels)

  • Sample Size Considerations:
      - Use rules of thumb for power calculations based on number of predictors (k)
      - Sample size of around 74 for overall multiple r, 107 for individual predictors

Data Cleaning and Outlier Examination

  • Essential to check for outliers in multiple regression
      - Multivariate outliers important due to multiple IVs

  • Mahalanobis distance for multivariate outlier detection:
      - Calculation involves distance from group centroid
      - Assessment through Chi-squared distribution with a critical alpha of 0.001

  • Decision-making with outliers:
      - Remove entire cases with problematic outliers
      - Conduct analyses with and without outlier cases to compare results

Regression Output Overview

  • Snapshot of regression output tables

  • Key elements include:
      - Model summary with multiple r
      - ANOVA table indicating significance of the model

  • Interpretation of beta weights alongside significance testing
      - Unstandardized and standardized beta weights bring clarity to predictor contributions

Beta Weights Interpretation

  • Unstandardized Beta Example:
      - Unit change in year results in increase in price
      - Flipping perspective for correct interpretation is necessary

  • Standardized Beta Weights:
      - Allow comparison of predictor strength across different units
      - Interpretation in standard deviation units for consistency

Correlation Types in Regression

  • Types of correlations obtained in multiple regression:
      - Zero-order correlation
      - Partial correlation
      - Semi-partial correlation (most relevant)

  • Importance of distinguishing between correlation types for thorough analysis

Regression Equation Formulation

  • Understanding the regression equation: y hat representation

  • Calculation of residuals:
      - Difference between observed score and predicted score
      - Residual as a measure of error in model predictions

Practical Application in Car Purchase

  • Data analysis application: predict value of potential car purchases using regression equation
      - Application involves evaluating multiple characteristics of cars

  • Decision-making based on calculated predictors reflecting market conditions

Generalizing Beyond Sample Data

  • Importance of generalizing models to broader populations
      - Example scenario of negotiating with a dealership using predictor weights
      - Comparison to actual market value based on calculated predictions

Inner Workings of Multiple Regression

  • Ordinary Least Squares (OLS) criterion for optimization
      - Minimizes sum of squared residuals through calculated predictions

  • Final insights and reflections on regression's capabilities

Conclusion

  • Wrap up of lecture contents and reinforcement of understanding multiple regression.

  • Call to apply concepts in practical situations beyond the classroom setting.