Chapter 2 notes: Ordinary least squares

Estimating Single-Independent-Variable Models with Ordinary Least Squares (OLS)

• Purpose of Regression Analysis:

  • To take a theoretical equation, such as: $(2.1) Y = \beta<em>0 + \beta</em>1X + \varepsilon$

  • And use empirical data to create an estimated equation: $(2.2) \hat{Y} = \hat{\beta}<em>0 + \hat{\beta}</em>1X$

• Ordinary Least Squares (OLS):

  • The most widely used method to obtain estimates for regression coefficients.

  • It has become the standard point of reference in econometrics.

• OLS Minimization Principle:

  • OLS calculates the estimated coefficients $\hat{\beta}<em>0$ and $\hat{\beta}</em>1$ by minimizing the sum of the squared residuals.

  • The residual ($e$) for each observation is the difference between the actual value of the dependent variable ($Y$) and its estimated value ($\hat{Y}$): $e = Y - \hat{Y}$

  • OLS Minimizes: $(2.3) \sum<em>{i=1}^{N} e</em>i^2$

  • Since $e<em>i = Y</em>i - \hat{Y}<em>i$ and $\hat{Y}</em>i = \hat{\beta}<em>0 + \hat{\beta}</em>1X_i$, Equation $(2.3)$ can be rewritten as:

    • OLS Minimizes: $\sum<em>{i=1}^{N} (Y</em>i - \hat{\beta}<em>0 - \hat{\beta}</em>1X_i)^2$

• Why Use OLS?

  • OLS is not the only regression estimation technique, but it is preferred for three main reasons:

    1. Ease of Use: OLS is relatively straightforward to implement and understand.

    2. Intuitive Goal: The objective of minimizing the sum of squared residuals has an intuitive appeal, as it aims to find the line that best fits the data by minimizing the total error.

    3. Desirable Properties:

       • The sum of the residuals ($\sum e_i$) for an OLS regression is exactly $0$.

       • Under certain classical assumptions (discussed in Chapter 4), OLS can be proven to be the