Topic 3 - Multiple Regression-Done

Page 1: Regression Analysis

• Topic: Multiple Regression

Page 2: Table of Contents

1. The Multiple Regression Model (Page 4)

2. Interpreting the Coefficients (Page 6)

3. Assessing the Model (Page 7)

   • 3.1 Test for Significance of Regression (Page 7)

   • 3.2 Test for Individual Regression Coefficients (t-tests) (Page 7)

   • 3.3 R² and Adjusted R² (Page 8)

4. Summary (Page 10)

Page 3: Introduction

• Regression Analysis Concept:

  • Multiple independent variables can provide information about the dependent variable being predicted.

  • The multiple regression model examines relationships between a dependent variable and multiple independent variables.

• Learning Objectives:

  • Explain the concept of multiple regression analysis.

  • Assess the utility of the multiple regression model.

Page 4: The Multiple Regression Model

• Definition:

  • Multiple regression relates one dependent variable (y) to multiple independent variables (x1, x2,..., xk).

• Example Application:

  • Sales influenced by variables like marketing budget, product price, quality, economy, and competition.

• Equation Structure:

  • Objective is to find an equation form that minimizes the sum of the squares of the errors (SSE).

• Hypothetical Example:

  • Sales (dependent) predicted by Price and Advertisement (independent variables).

Page 5: Data Example

• Table 1: Sales of a Company

  Sales (US $ ‘000)	Price (US $)	Advertisement (US $ ‘000)
  88	117	33
  90	111	37
  100	120	34
  ...	...	...

• Assumption:

  • Linear relationships between sales, price, and advertisement justify using multiple linear regression.

Page 6: Interpreting the Coefficients

• Interpretation Approach:

  • Similar to simple linear regression with careful distinction for each coefficient.

• Case Example:

  • Sample regression equation like: y = b0 + b1x1 + b2x2

  • Coefficient b0 indicates the intercept when both independent variables x1 and x2 are zero (less relevant in practice).

  • Slope coefficients (b1, b2) denote the increase in the dependent variable with unit increase in the respective independent variable, holding others constant.

Page 7: Assessing the Model

3.1 Test for Significance of Regression

• Hypotheses:

  • H0: β1 = β2 …βk = 0 (No effect)

  • H1: At least one βi is not equal to zero (At least one variable is effective)

• F-Test:

  • P-value indicates the model's usefulness:

    • Not rejected: Model has limited usefulness.

    • 
      

      Rejected: At least one independent variable is significant.3.2 Test for Individual Regression Coefficients (t-tests)

• Explore independent variables' individual significance via t-tests from ANOVA section.

• Null and Alternative hypothesis for individual variables:

  • H0: βi = 0

  • 
    

    H1: βi ≠ 03.3 R² and Adjusted R²

• R²: Reflects proportion of variance explained, can increase with irrelevant variables.

• Adjusted R²: Accounts for degrees of freedom, penalizes irrelevant variables for more accurate assessment.

Page 8: Effects of R² vs. Adjusted R²

• Example of R² Misrepresentation:

  • An increase in R² when adding an irrelevant independent variable can mislead about model fit.

  • Always consider Adjusted R² alongside R² for a more accurate representation of regression quality.

• Case Stats:

  • Example showed R² at 0.94, Adjusted R² at 0.92.

Page 9: Summary

• Key Learning Points:

  • Multiple regression relates a dependent variable to multiple independent variables.

  • Use F-test for overall model assessment and individual t-tests for assessing variable significance.