Gujarati Porter-2010-Essentials-CH-02

CHAPTER 2: BASIC IDEAS OF LINEAR REGRESSION: THE TWO-VARIABLE MODEL

Introduction to Regression Analysis

  • Regression analysis is a statistical technique focusing on the relationship between a dependent variable and one or more independent variables.

  • This chapter introduces the fundamentals of regression analysis starting with the simplest linear regression model: the two-variable model.

  • Subsequent sections will explore variations and extensions of this foundational model.

2.1 The Meaning of Regression

  • Regression analysis examines how one variable (dependent) relates to others (independent).

  • Common applications:

    • Quantity demanded of a commodity vs. its price, consumer income, and prices of competing goods.

    • Sales of a product related to advertising expenditure.

    • Analysis of defense spending vs. gross domestic product (GDP).

  • Notation: From now on, let Y be the dependent variable and X be the independent variable. For additional Xs, denote them with subscripts (X1, X2, etc.).

  • Causation Warning: Regression does not imply causation; causal relationships must be supported by underlying theory (e.g., the law of demand explains how quantity demanded is influenced by price).

2.2 Population Regression Function (PRF)

  • The PRF is an expression that captures the average (mean) value of Y based on values of X.

  • Example: Y (Math S.A.T. score) depends on X (family income) for students.

  • Income divided into classes (e.g., $0-$10,000, $10,000-$20,000, etc.). Midpoints are utilized for analysis.

  • As income increases, the average math score (Y) tends to increase, suggesting a positive relationship.

  • PRL: The Population Regression Line represents the average value of Y across all values of X.


  • The PRF mathematically can be represented as:E(Y|Xi) = B1 + B2Xi

2.3 Statistical Specification of PRF

  • Stochastic Error Term (u): Unpredictable factors influencing individual scores.

  • Individual math scores vary from their respective group means due to the error term.

  • The stochastic PRF reflects deviations from the mean caused by the error term: Yi = B1 + B2Xi + ui

  • Role of the Error Term (u):

    • Accounts for unmeasured variables affecting Y (e.g., GPA, additional inequalities).

    • Captures inherent variability (randomness) in human behavior.

    • Represents measurement errors, such as inaccuracies in income reporting.

2.4 Sample Regression Function (SRF)

  • In practice, we often only have sample data, not the entire population. Thus, we use the SRF: Yi = b1 + b2Xi + ei

  • Residual (ei): Difference between actual data points and the estimated values from the SRF.

  • The goal is to estimate B1 and B2 so that residuals are minimized (Ordinary Least Squares method).

2.5 Ordinary Least Squares (OLS) Estimation

  • Objective: Minimize the residual sum of squares (RSS).

  • The estimators can be derived using simultaneous equations:

    • b1 = sample mean of Y - b2 * sample mean of X

    • b2 = covariance of X and Y divided by the variance of X.

2.6 Special Meaning of "Linear" Regression

  • Linearity in Variables: The mean value of Y must be a linear function of X.

    • Nonlinear functions do not maintain a constant slope.

  • Linearity in Parameters: A model is linear in parameters if parameters appear only to the first power.

2.7 Two-Variable vs. Multiple Linear Regression

  • The discussion may extend to models with multiple independent variables, enhancing predictive power and explanation of the dependent variable's variation.

2.8 Summary of Key Concepts

  • Regression analysis fundamental formulas, notation, and concepts have been detailed to aid understanding of relationships between variables.

  • Understanding the distinctions and calculations involved lays the groundwork for more complex analysis in future chapters.

Key Terms and Concepts

  • Regression Analysis, Explained Variable, Independent Variable, Population Regression Line (PRL), Sample Regression Function (SRF), Stochastic Error Term, Residual Term, Ordinary Least Squares (OLS).

Practice Questions

  • Questions focus on the application and understanding of the concepts discussed, encouraging deep engagement with the material and preparation for examinations.