1/26
Looks like no tags are added yet.
Which objects are observed, which are unobserved, and which are computed from the sample?
y, x observed (sample); β₀, β₁, u unobserved (population); β̂₀, β̂₁, ŷᵢ, ûᵢ and the OLS line computed/estimated (sample)
y = dependent (explained) variable. x = independent (explanatory) variable, or regressor. β₀, β₁ = parameters (intercept, slope). u = error term. β̂ = OLS estimates. ŷᵢ = fitted (predicted) value. ûᵢ = residual = actual − fitted. ŷ = β̂₀ + β̂₁x is the OLS regression line (sample regression function). Hats = estimated/computed from the sample. The residual ûᵢ is computed, but the error uᵢ is never observed.
Software computes the OLS estimates, so why derive them yourself?
Software computes the algebra that comes from theory, but it won't tell you whether the assumptions are plausible or whether your inference is valid and sensible
R solves min over β̂₀, β̂₁ of Σ(yᵢ − β̂₀ − β̂₁xᵢ)². Understanding where the numbers come from tells you what they mean and when to trust them.
Which three calculus rules do you need to derive OLS?
(1) Differentiate a sum term by term; (2) chain rule for squares: ∂(stuff)²/∂β = 2·stuff·∂(stuff)/∂β; (3) anything without the β you're differentiating is a constant, e.g. ∂(yᵢ − β₁xᵢ)/∂β₁ = −xᵢ
Applying these to S = Σ(yᵢ − β₀ − β₁xᵢ)² gives ∂S/∂β₀ = −2Σ(yᵢ − β̂₀ − β̂₁xᵢ) and ∂S/∂β₁ = −2Σxᵢ(yᵢ − β̂₀ − β̂₁xᵢ). Set both to 0, giving FOC 1 and FOC 2.
Which algebraic properties come from the two first-order conditions?
FOC 1 (intercept): Σûᵢ = 0 and the line passes through (x̄, ȳ). FOC 2 (slope): Σxᵢûᵢ = 0, which together with FOC 1 means the residuals have zero sample covariance with x
Why 'given FOC 1': the sample covariance is (1/n)Σ(xᵢ − x̄)(ûᵢ − ū) = (1/n)[Σxᵢûᵢ − x̄Σûᵢ], because ū = 0. Both terms are 0. These properties hold in every OLS regression with an intercept; they say nothing about how good the model is.
How do we know the OLS solution is a minimum, and that it's unique?
S is a convex quadratic. The Hessian has a positive diagonal (2n > 0) and a positive determinant (4nΣ(xᵢ − x̄)² > 0, provided x varies), so there is exactly one minimum
The second derivatives are constants: ∂²S/∂β₀² = 2n, ∂²S/∂β₁² = 2Σxᵢ², ∂²S/∂β₀∂β₁ = 2Σxᵢ. det H = 4nΣxᵢ² − 4(Σxᵢ)² = 4nΣ(xᵢ − x̄)² > 0 when x varies. A positive diagonal plus a positive determinant means H is positive definite, so the function is convex and has a unique minimum. That's why exactly one line minimises the SSR.
You regress hourly wage ($) on years of education and get wage-hat = −0.90 + 0.54·educ. How do you interpret the slope correctly?
"In this sample, one more year of education is associated with a $0.54 higher predicted hourly wage."
Four key elements: 'associated with', not 'increases' (no causal claims yet); 'predicted', because it's a point on the line, not any individual worker; units of y (dollars per hour); 'in this sample' (the 526 workers we have). Terminology: we 'regress Y on X'.
How do you interpret the intercept −0.90 in wage-hat = −0.90 + 0.54·educ?
The predicted hourly wage at zero years of education is −$0.90. It has no useful economic meaning: a negative wage is impossible, and only 2 of 526 workers have zero education, so the line is a poor guide there.
The intercept is always the predicted y at x = 0. Whether that means anything depends on whether x = 0 is realistic and well covered by the data. Extrapolating the line outside the data range is unreliable.
Using wage-hat = −0.90 + 0.54·educ: what is the fitted value for 12 years of education, and the residual for a worker with 12 years who earns $7.00?
Fitted ≈ $5.59; residual = 7.00 − 5.59 = $1.41 (she earns $1.41 more than predicted)
Fitted: −0.90 + 0.54(12) = 5.58 using the rounded coefficients (R reports 5.59 from the unrounded ones). Residual = actual − fitted. Positive means the point is above the line (earns more than predicted); negative means below. The residual is in the same units as y (dollars per hour). At 16 years: −0.90 + 0.54(16) ≈ $7.76.
Spot the error: "An extra year of education INCREASES a worker's wage by 0.54."
It's causal language, and it has no units or 'predicted'. It should say 'is associated with a $0.54 higher predicted hourly wage'
With observational data, SLR describes an association. Causality needs extra assumptions (Part II of the course). Also state the units (dollars per hour) and that it's the predicted (average) wage, not any individual worker's.
Spot the error: "An extra year of education is associated with 0.54% higher wages."
Wrong units: in a level-level model the slope is in units of y (dollars), not percent. A percentage reading needs log(y) on the left-hand side
Level-level: Δy = β₁Δx in units of y. Only in a log-level model does %Δy ≈ 100·β₁·Δx.
Spot the error: "A worker with no education earns −$0.90 an hour."
It's a predicted value from the line, not what an actual worker earns. It's also extrapolation to a region with almost no data, giving an impossible negative wage
Correct version: 'The predicted hourly wage at zero years of education is −$0.90, which has no useful economic meaning.'
Spot the error: "The residuals sum to zero, so the model fits well."
Σûᵢ = 0 holds automatically in every OLS regression with an intercept (it comes from FOC 1), so it says nothing about fit. Fit is measured by R²
It's an algebraic property, not a measure of quality. Large positive and negative residuals can cancel out.
Define SST, SSE and SSR (Wooldridge's naming).
SST = Σ(yᵢ − ȳ)² (total); SSE = Σ(ŷᵢ − ȳ)² (explained); SSR = Σûᵢ² (residual)
Each deviation splits as yᵢ − ȳ = (ŷᵢ − ȳ) + ûᵢ, i.e. total = explained part + residual part. Stock & Watson call these TSS, ESS and SSR: same objects, different names. Watch out: SSE means 'explained' in Wooldridge but 'error' in some other books.
Prove SST = SSE + SSR. Which step makes it work?
Write yᵢ − ȳ = (ŷᵢ − ȳ) + ûᵢ, square and sum. The cross term 2Σûᵢ(ŷᵢ − ȳ) = 0 because FOC 1 (Σûᵢ = 0) and FOC 2 (Σxᵢûᵢ = 0) kill it
Σ(yᵢ − ȳ)² = Σ(ŷᵢ − ȳ)² + Σûᵢ² + 2Σûᵢ(ŷᵢ − ȳ). Cross term: ŷᵢ − ȳ = β̂₀ + β̂₁xᵢ − ȳ, so Σûᵢ(ŷᵢ − ȳ) = (β̂₀ − ȳ)Σûᵢ + β̂₁Σxᵢûᵢ = 0 + 0. FOC 1 kills the first term and FOC 2 kills the second. You need an intercept in the model for this (otherwise FOC 1 doesn't hold).
What is R², what range does it lie in, and how does it relate to correlation?
R² = SSE/SST = 1 − SSR/SST: the share of sample variation in y explained by the fitted values. It lies between 0 and 1, and in SLR with an intercept it equals the squared sample correlation of x and y
wage on educ, n = 526: R² = 0.165. How to say it: 'In this sample, the regression on education explains 16.5% of the variation in hourly wages.' 'Explained' is a statistical term, not a causal one.
What does a low or a high R² tell you about the slope?
A low R² doesn't make the slope useless; it just means most of the variation in y isn't captured by the line. A high R² doesn't make the slope causal; fit describes the sample, it doesn't explain why
R² measures goodness of fit only. It is separate from causality, bias and significance.
What happens to the coefficients and R² if you multiply y by a constant c (e.g. wage in cents, c = 100)?
Both β̂₀ and β̂₁ are multiplied by c; R² is unchanged
Cents: intercept −90.5 and slope 54.1 (vs −0.90 and 0.54), R² still 0.165. Why: if y* = cy then ȳ* = cȳ and Sxy* = cSxy, so β̂₁* = cβ̂₁ and β̂₀* = cȳ − cβ̂₁x̄ = cβ̂₀. The fitted values and mean scale by c, so SSE and SST both scale by c² and the ratio is unchanged.
What happens to the coefficients and R² if you multiply x by a constant d (e.g. education in months, d = 12)?
The slope is divided by d; the intercept and R² are unchanged
Months: slope = 0.54/12 = 0.045, intercept −0.90, R² 0.165. How to say it: 'One more month of education is associated with a $0.045 higher predicted hourly wage.' Why: x* = dx gives Sxx* = d²Sxx and Sxy* = dSxy, so β̂₁* = β̂₁/d. Then β̂₀* = ȳ − (β̂₁/d)(dx̄) = β̂₀. The fitted values are unchanged. Rule: units change the coefficients, not the fit, because R² has no units.
Why use the (natural) log of wages?
Wages are right-skewed (long right tail), and log wages are much closer to symmetric. Logs also make the slope describe proportional (%) rather than absolute differences
log = natural log (ln). Percentage effects are often more economically sensible, e.g. a year of education raising wages by a constant % rather than a constant number of dollars.
Log-level model: log(wage)-hat = β̂₀ + 0.0827·educ. How do you interpret 0.0827?
"In this sample, one more year of education is associated with an approximately 8.3% higher predicted hourly wage" (100 × 0.0827)
Rule: %Δy ≈ (100·β₁)·Δx. Why: log y₁ − log y₀ = β₁Δx, so y₁/y₀ = exp(β₁Δx). The exact change is %Δy = 100·[exp(β₁Δx) − 1], and since exp(z) − 1 ≈ z for small z, %Δy ≈ 100·β₁Δx. The approximation is good when β̂₁Δx is small.
Exact vs approximate % change: for slopes of 0.0827 and −0.397, what are the exact and approximate effects?
0.0827: exact 8.6%, approx 8.3%. −0.397: exact −32.8%, approx −39.7%
Exact = 100·(exp(β̂₁) − 1): 100(e^0.0827 − 1) = 8.6% and 100(e^−0.397 − 1) = −32.8%. The approximation 100·β̂₁ is fine for small coefficients but poor for large ones. Use the exact formula when |β̂₁| is large (roughly above 0.1 to 0.2).
Level-log model y = β₀ + β₁log(x) + u: how do you read β₁?
Δy ≈ (β₁/100)·%Δx: a 1% increase in x is associated with a β₁/100 unit change in y
Example: if β̂₁ = 50 and y is in £, a 1% rise in x is associated with a £0.50 higher predicted y. Units are those of y.
Log-log model log y = β₀ + β₁log(x) + u: how do you read β₁?
As an elasticity: %Δy ≈ β₁·%Δx, so a 1% rise in x is associated with a β₁% change in y
Example: β̂₁ = 0.3 means a 1% rise in x is associated with an approximately 0.3% higher predicted y. Summary of all four forms: level-level, Δy = β₁Δx (units of y); log-level, %Δy ≈ 100β₁Δx; level-log, Δy ≈ (β₁/100)%Δx; log-log, %Δy ≈ β₁%Δx.
Can you compare the R² of a model with y against one with log(y)?
No. They measure explained variation on different scales (variation in y vs in log y)
R² is unit-free under rescaling (multiplying by a constant), but taking logs is a non-linear transformation that changes what 'total variation' means. You can only compare R² across models with the same dependent variable.
What is the mean of a 0/1 (dummy) variable?
The share (proportion) of observations equal to 1
D̄ = (1/n)ΣDᵢ. The sum counts the 1s (0s add nothing), and dividing by n gives the proportion. E.g. a mean of female = 0.479 means 47.9% of the sample is female.
In y = β₀ + β₁D + u, with D = 1 for women and 0 for men, what do the OLS estimates equal?
β̂₀ = ȳ₀ (mean of y for D = 0, men); β̂₁ = ȳ₁ − ȳ₀ (women's mean minus men's mean)
Proof via the FOCs. FOC 2: ΣDᵢûᵢ = 0 only adds residuals where D = 1, so women's residuals sum to 0, which gives ȳ₁ = β̂₀ + β̂₁. FOC 1: Σûᵢ = 0, so the men's residuals must also sum to 0, which gives ȳ₀ = β̂₀. Hence β̂₁ = ȳ₁ − ȳ₀. The D = 0 group is the reference (base) group.
wage-hat = 7.10 − 2.51·female. Interpret both coefficients and give women's mean wage.
7.10 = men's mean hourly wage; −2.51 = women's mean minus men's; women's mean = 7.10 − 2.51 = $4.59. "In this sample, women's mean hourly wage is $2.51 lower than men's"
Don't make it causal ('being female lowers wages'). The regression recovers MEANS, not medians (the medians in Practical 1's box plot were $6.00 and $3.75).