Gujarati Porter-2010-Essentials-CH-02
CHAPTER 2: BASIC IDEAS OF LINEAR REGRESSION: THE TWO-VARIABLE MODEL
Introduction to Regression Analysis
Regression analysis is a statistical technique focusing on the relationship between a dependent variable and one or more independent variables.
This chapter introduces the fundamentals of regression analysis starting with the simplest linear regression model: the two-variable model.
Subsequent sections will explore variations and extensions of this foundational model.
2.1 The Meaning of Regression
Regression analysis examines how one variable (dependent) relates to others (independent).
Common applications:
Quantity demanded of a commodity vs. its price, consumer income, and prices of competing goods.
Sales of a product related to advertising expenditure.
Analysis of defense spending vs. gross domestic product (GDP).
Notation: From now on, let Y be the dependent variable and X be the independent variable. For additional Xs, denote them with subscripts (X1, X2, etc.).
Causation Warning: Regression does not imply causation; causal relationships must be supported by underlying theory (e.g., the law of demand explains how quantity demanded is influenced by price).
2.2 Population Regression Function (PRF)
The PRF is an expression that captures the average (mean) value of Y based on values of X.
Example: Y (Math S.A.T. score) depends on X (family income) for students.
Income divided into classes (e.g., $0-$10,000, $10,000-$20,000, etc.). Midpoints are utilized for analysis.
As income increases, the average math score (Y) tends to increase, suggesting a positive relationship.
PRL: The Population Regression Line represents the average value of Y across all values of X.
The PRF mathematically can be represented as:E(Y|Xi) = B1 + B2Xi
2.3 Statistical Specification of PRF
Stochastic Error Term (u): Unpredictable factors influencing individual scores.
Individual math scores vary from their respective group means due to the error term.
The stochastic PRF reflects deviations from the mean caused by the error term: Yi = B1 + B2Xi + ui
Role of the Error Term (u):
Accounts for unmeasured variables affecting Y (e.g., GPA, additional inequalities).
Captures inherent variability (randomness) in human behavior.
Represents measurement errors, such as inaccuracies in income reporting.
2.4 Sample Regression Function (SRF)
In practice, we often only have sample data, not the entire population. Thus, we use the SRF: Yi = b1 + b2Xi + ei
Residual (ei): Difference between actual data points and the estimated values from the SRF.
The goal is to estimate B1 and B2 so that residuals are minimized (Ordinary Least Squares method).
2.5 Ordinary Least Squares (OLS) Estimation
Objective: Minimize the residual sum of squares (RSS).
The estimators can be derived using simultaneous equations:
b1 = sample mean of Y - b2 * sample mean of X
b2 = covariance of X and Y divided by the variance of X.
2.6 Special Meaning of "Linear" Regression
Linearity in Variables: The mean value of Y must be a linear function of X.
Nonlinear functions do not maintain a constant slope.
Linearity in Parameters: A model is linear in parameters if parameters appear only to the first power.
2.7 Two-Variable vs. Multiple Linear Regression
The discussion may extend to models with multiple independent variables, enhancing predictive power and explanation of the dependent variable's variation.
2.8 Summary of Key Concepts
Regression analysis fundamental formulas, notation, and concepts have been detailed to aid understanding of relationships between variables.
Understanding the distinctions and calculations involved lays the groundwork for more complex analysis in future chapters.
Key Terms and Concepts
Regression Analysis, Explained Variable, Independent Variable, Population Regression Line (PRL), Sample Regression Function (SRF), Stochastic Error Term, Residual Term, Ordinary Least Squares (OLS).
Practice Questions
Questions focus on the application and understanding of the concepts discussed, encouraging deep engagement with the material and preparation for examinations.