Multiple regression

Multiple Linear Regression, Part 1

Outline

• Multiple linear regression models
• Least squares estimation
• Fitted values, residuals, estimate of variance
• Interpretation of regression coefficients

What are Multiple Linear Regression Models

Deterministic Models (No Errors)

• Deterministic models describe perfect relationships between variables without errors: $Y = f(X<em>1, X</em>2, …, X_p)$

  • Examples:

  • Newton’s second law of motion: $F = m \times a$ (Force = mass × acceleration)

  • Ideal gas law: $PV = nRT$ (pressure × volume = amount of gas in moles × ideal gas constant × temperature in °K)

Timber Volume of Trees

• Modeling timber volume of a tree as a function of its radius and height:

  • If the trunk is a cylinder: $volume = \pi r^2 h$, where r = radius, h = height
  • If the trunk is a cone: $volume = \frac{1}{3} \pi r^2 h$
  • Tree trunks are not exactly cylinders or cones, so the formulas are subject to error.
  • Model with error:

$volume = f(r, h) + \epsilon = \alpha r^2 h + \epsilon$, where $\alpha$ is a constant.

Statistical Models

• A statistical model is a simple, low-dimensional summary of:

  • The relationship present in the data
  • The data-generation process
  • The relationship present in the population

• Statistical models allow errors (uncertainty): $Y = f(X<em>1, X</em>2, …, X_p) + \epsilon$

  • Y = response
  • f(X1, X2, . . . , Xp) = deterministic function
  • ε = error (noise)

Linear Regression Models

• Focus on linear regression models: $Y = f(X<em>1, X</em>2, …, X*p) + \epsilon = \beta<em>0 + \beta</em>1 X<em>1 + \beta</em>2 X<em>2 + … + \beta</em>p X_p + \epsilon$
  • Linearity means the model is linear in its parameters $\beta<em>0, \beta</em>1, …, \beta_p$.
  • Examples of linear regression models:
  • $Y = \beta<em>0 + \beta</em>1 X + \beta_2 X^2 + \epsilon$
  • $Y = \beta<em>0 + \beta</em>1 log(X) + \epsilon$
  • Even though the relationship between Y and X is not linear.

Some Non-linear Models Can Be Turned Linear (1)

• Ex 1: Reciprocal Transformation

  • Non-linear model: $Y = \frac{X}{\alpha X + \beta}$

  • Transformed:

$\frac{1}{Y} = \alpha + \beta(\frac{1}{X})$

• Linear model: $Y' = \alpha + \beta X'$, where $Y' = \frac{1}{Y}, X' = \frac{1}{X}$.

Some Non-linear Models Can Be Turned Linear (2)

• Ex 2: Timber volume of trees $≈ cr^2h$ or more generally, $\alpha r^{\beta<em>1} h^{\beta</em>2}$

  • Non-linear model: $Volume = \alpha \times r^{\beta<em>1} \times h^{\beta</em>2}$
  • Taking logarithm: $log(Volume) = log(\alpha) + \beta<em>1 log(r) + \beta</em>2 log(h)$
  • Linear model: $Y = \beta<em>0 + \beta</em>1 X<em>1 + \beta</em>2 X<em>2$ where $Y = log(Volume)$, $X</em>1 = log(r) = log(radius)$, and $X_2 = log(h) = log(height)$.

Some Non-linear Models Can Be Turned Linear (3)

• Ex 3: Cobb-Douglas Production Function

  • In economics, the Cobb-Douglas production function is:

$V = \alpha K^{\beta<em>1} L^{\beta</em>2}$, where K = capital, L = labor

• Represents the relationship between inputs (capital K and labor L) and output V.
• Linear transformation:

$log(V) = log(\alpha) + \beta<em>1 log(K) + \beta</em>2 log(L)$.

Identifying Linear Models

• Determining which of the following models are linear:

  • (a) $Y = \beta<em>0 + \beta X</em>1 + \epsilon$ (Linear)
  • (b) $Y = \beta<em>0 \beta^{X</em>1} \epsilon$ (Non-linear)
  • (c) $Y = \beta<em>0 + \beta</em>1 e^X + \epsilon$ (Linear)
  • (d) $Y = \beta<em>0 + \beta</em>1 X^2 + \beta_2 log(X) + \epsilon$ (Linear)

Identifying Transformable Models

• Determining which models can be turned linear after transformation:

  • (a) $Y = \beta<em>0 + \beta^{X</em>1} + \epsilon$
  • Not transformable
  • (b) $Y = \beta<em>0 \beta^{X</em>1} \epsilon$
  • Transformable

Data for Multiple Linear Regression Models

• SLR observes pairs of data values.
• MLR observes rows of data values.
• Each row (or pair) is a case, a record, or a data point.
• $y_i$ is the response (dependent variable) of the $i^{th}$ case.
• There are p explanatory variables (predictors, covariates), and $x<em>{ik}$ is the value of the explanatory variable $X</em>k$ of the $i^{th}$ case.

Multiple Linear Regression Models

$y<em>i = \beta</em>0 + \beta<em>1 x</em>{i1} + … + \beta<em>p x</em>{ip} + \epsilon_i$

• where:

  • $\epsilon_i$’s (errors, or noise) are i.i.d. $N(0, \sigma^2)$
  • Parameters include:
  • $\beta_0$ = intercept;
  • $\beta_k$ = regression coefficient (slope) for the $k^{th}$ explanatory variable, k = 1, . . . , p
  • $\sigma^2$ = Var($\epsilon_i$) = the variance of errors
  • Observed (known): $y<em>i, x</em>{i1}, x<em>{i2}, …, x</em>{ip}$
  • Unknown: $\beta<em>0, \beta</em>1, …, \beta<em>p, \sigma^2 , \epsilon</em>i$’s
  • Random: $\epsilon<em>i$’s, $y</em>i$’s
  • Constants (not random): $\beta<em>k$’s, $\sigma^2$, $x</em>{ik}$’s

Multiple Linear Regression Models in Matrix Notation

• Matrix notation:

$Y<em>{n \times 1} = \begin{bmatrix} y</em>1 \ y<em>2 \ … \ y</em>n \end{bmatrix}$,

$X<em>{n \times (p+1)} = \begin{bmatrix} 1 & x</em>{11} &amp; x<em>{21} & \cdots & x</em>{p1} \ 1 &amp; x<em>{12} & x</em>{22} &amp; \cdots &amp; x<em>{p2} \ … & … & … & … & … \ 1 & x</em>{1n} &amp; x<em>{2n} & \cdots & x</em>{pn} \end{bmatrix}$,

$\beta<em>{(p+1) \times 1} = \begin{bmatrix} \beta</em>0 \ \beta<em>1 \ \beta</em>2 \ … \ \beta<em>p \end{bmatrix}$, $\epsilon</em>{n \times 1} = \begin{bmatrix} \epsilon<em>1 \ \epsilon</em>2 \ … \ \epsilon_n \end{bmatrix}$

• Model: $Y = X\beta + \epsilon$

Least Squares Estimation

Fitting the Mode

Least Squares Method (SLR)

• Least squares estimate $\hat{\beta<em>0}, \hat{\beta</em>1}$ for $(\beta<em>0, \beta</em>1)$ is the intercept and slope of the straight line with the minimum sum of squared vertical distances to the data points:

$\sum<em>{i=1}^{n}(y</em>i - \hat{\beta<em>0} - \hat{\beta</em>1} x_i)^2$

Least Squares Method (MLR)

• The least squares estimate $(\hat{\beta<em>0}, …, \hat{\beta</em>p})$ for $(\beta<em>0, …, \beta</em>p)$ is the intercept and slopes of the (hyper)plane with the minimum sum of squared vertical distance to the data points:

$\sum<em>{i=1}^{n}(y</em>i - \hat{\beta<em>0} - \hat{\beta</em>1} x<em>{i1} - … - \hat{\beta</em>p} x_{ip})^2$

The “Hat” Notation:

• Differentiate:

  • Estimated coefficient $\hat{\beta_j}$ from
  • The actual unknown coefficient $\beta_j$

Least Squares Problem for SLR

• Minimize $L(\hat{\beta<em>0}, \hat{\beta</em>1}) = \sum<em>{i=1}^{n} (y</em>i - \hat{\beta<em>0} - \hat{\beta</em>1} x_i)^2$
• Set derivatives to 0:

$\frac{\partial L}{\partial \hat{\beta<em>0}} = -2\sum</em>{i=1}^{n}(y<em>i - \hat{\beta</em>0} - \hat{\beta<em>1} x</em>i) = 0$

$\frac{\partial L}{\partial \hat{\beta<em>1}} = -2\sum</em>{i=1}^{n} x<em>i(y</em>i - \hat{\beta<em>0} - \hat{\beta</em>1} x_i) = 0$

• This results in the 2 equations below in 2 unknowns $\hat{\beta<em>0}$ and $\hat{\beta</em>1}$.

$n\hat{\beta<em>0} + \hat{\beta</em>1}\sum<em>{i=1}^{n} x</em>i = \sum<em>{i=1}^{n} y</em>i$

$\hat{\beta<em>0}\sum</em>{i=1}^{n} x<em>i + \hat{\beta</em>1}\sum<em>{i=1}^{n} x</em>i^2 = \sum<em>{i=1}^{n} x</em>i y_i$

Solving for Estimates

• From the normal equations:

$n\hat{\beta<em>0} + \hat{\beta</em>1} \sum<em>{i=1}^{n} x</em>i = \sum<em>{i=1}^{n} y</em>i$

$\hat{\beta<em>0} \sum</em>{i=1}^{n} x<em>i + \hat{\beta</em>1} \sum<em>{i=1}^{n} x</em>i^2 = \sum<em>{i=1}^{n} x</em>i y_i$

• Rewrite using means:

$n\hat{\beta<em>0} + \hat{\beta</em>1} n\bar{x} = n\bar{y}$

$\hat{\beta<em>0} n\bar{x} + \hat{\beta</em>1} \sum<em>{i=1}^{n} x</em>i^2 = \sum<em>{i=1}^{n} x</em>i y_i$

• Divide the first equation by n:

$\hat{\beta<em>0} + \hat{\beta</em>1} \bar{x} = \bar{y} \implies \hat{\beta<em>0} = \bar{y} - \hat{\beta</em>1} \bar{x}$

• Substitute into the second equation:

$\hat{\beta<em>0} n\bar{x} + \hat{\beta</em>1} \sum<em>{i=1}^{n} x</em>i^2 = \sum<em>{i=1}^{n} x</em>i y_i$

$(\bar{y} - \hat{\beta<em>1} \bar{x})n\bar{x} + \hat{\beta</em>1} \sum<em>{i=1}^{n} x</em>i^2 = \sum<em>{i=1}^{n} x</em>i y_i$

• Solve for $\hat{\beta<em>1}$: $\hat{\beta</em>1} (\sum<em>{i=1}^{n} x</em>i^2 - n\bar{x}^2) = \sum<em>{i=1}^{n} x</em>i y_i - n\bar{x}\bar{y}$

$\hat{\beta<em>1} = \frac{\sum</em>{i=1}^{n} x<em>i y</em>i - n\bar{x}\bar{y}}{\sum<em>{i=1}^{n} x</em>i^2 - n\bar{x}^2}$

Homework

• Show that

$\sum<em>{i=1}^{n}(x</em>i - \bar{x})(y<em>i - \bar{y}) = \sum</em>{i=1}^{n}(x<em>i - \bar{x})y</em>i = \sum<em>{i=1}^{n} x</em>i y_i - n\bar{x}\bar{y}$.

• Show that

$\sum<em>{i=1}^{n}(x</em>i - \bar{x})^2 = \sum<em>{i=1}^{n} x</em>i^2 - n\bar{x}^2$

• Hence, there are 3 formulae for LS estimate of the slope:
  • $\hat{\beta<em>1} = \frac{\sum</em>{i=1}^{n} x<em>i y</em>i - n\bar{x}\bar{y}}{\sum<em>{i=1}^{n} x</em>i^2 - n\bar{x}^2} = \frac{\sum<em>{i=1}^{n}(x</em>i - \bar{x})(y<em>i - \bar{y})}{\sum</em>{i=1}^{n}(x<em>i - \bar{x})^2} = \frac{\sum</em>{i=1}^{n}(x<em>i - \bar{x})y</em>i}{\sum<em>{i=1}^{n}(x</em>i - \bar{x})^2}$

Least Squares Problem for MLR

• To find the $(\hat{\beta<em>0}, \hat{\beta</em>1}, …, \hat{\beta<em>p})$ that minimize $L(\hat{\beta</em>0}, \hat{\beta<em>1}, …, \hat{\beta</em>p}) = \sum<em>{i=1}^{n} (y</em>i - \hat{\beta<em>0} - \hat{\beta</em>1} x<em>{i1} - … - \hat{\beta</em>p} x_{ip})^2$
• Set the derivatives of L with respect to $\hat{\beta<em>j}$ to 0 $\frac{\partial L}{\partial \hat{\beta</em>0}} = -2\sum<em>{i=1}^{n} (y</em>i - \hat{\beta<em>0} - \hat{\beta</em>1} x<em>{i1} - … - \hat{\beta</em>p} x<em>{ip})$$\frac{\partial L}{\partial \hat{\beta</em>k}} = -2\sum<em>{i=1}^{n} x</em>{ik}(y<em>i - \hat{\beta</em>0} - \hat{\beta<em>1} x</em>{i1} - … - \hat{\beta<em>p} x</em>{ip}), k = 1, 2, …, p$
• This results in a system of $(p + 1)$ equations in $(p + 1)$ unknowns (normal equations).

Least Squares Problem for MLR: The Normal Equations

• The least squares estimate $(\hat{\beta<em>0}, \hat{\beta</em>1}, …, \hat{\beta_p})$ is the solution to the following system of equations, called the normal equations.
  • $\hat{\beta<em>0} \cdot n + \hat{\beta</em>1} \sum<em>{i=1}^{n} x</em>{i1} + \cdots + \hat{\beta<em>p} \sum</em>{i=1}^{n} x<em>{ip} = \sum</em>{i=1}^{n} y_i$

$\hat{\beta<em>0} \sum</em>{i=1}^{n} x<em>{i1} + \hat{\beta</em>1} \sum<em>{i=1}^{n} x</em>{i1}^2 + \cdots + \hat{\beta<em>p} \sum</em>{i=1}^{n} x<em>{i1} x</em>{ip} = \sum<em>{i=1}^{n} x</em>{i1} y_i$

$\vdots$

$\hat{\beta<em>0} \sum</em>{i=1}^{n} x<em>{ik} + \hat{\beta</em>1} \sum<em>{i=1}^{n} x</em>{ik} x<em>{i1} + \cdots + \hat{\beta</em>p} \sum<em>{i=1}^{n} x</em>{ik} x<em>{ip} = \sum</em>{i=1}^{n} x<em>{ik} y</em>i$

$\vdots$

$\hat{\beta<em>0} \sum</em>{i=1}^{n} x<em>{ip} + \hat{\beta</em>1} \sum<em>{i=1}^{n} x</em>{ip} x<em>{i1} + \cdots + \hat{\beta</em>p} \sum<em>{i=1}^{n} x</em>{ip}^2 = \sum<em>{i=1}^{n} x</em>{ip} y_i$

• In matrix notation, the normal equation is $(X^T X)\hat{\beta} = X^T Y$, and the least squares estimate is $\hat{\beta} = (X^T X)^{-1}X^T Y$
• Don’t worry about solving the equations. R and other software can do the computation for us.

Parameters vs. Estimates

• $\beta<em>i$’s are the coefficients of the MLR model, and $\hat{\beta</em>i}$’s are the estimates of $\beta_i$’s.
• For SLR model:
  • $y = \beta<em>0 + \beta</em>1 x$ is the least square line for the population.
  • $y = \hat{\beta<em>0} + \hat{\beta</em>1} x$ is the least square line for a sample.
• Population:
  • $y = \beta<em>0 + \beta</em>1 x$: least-square regression line of the population, fixed, unknown, not of interest.
• Sample:
  • $y = \hat{\beta<em>0} + \hat{\beta</em>1} x$: least-square regression line of the sample, random, changes from sample to sample, can be calculated from sample of interest

Fitted Values, Residuals, Estimate of $\sigma^2$

Fitted Values

• The fitted value or predicted value:

$\hat{y<em>i} = \hat{\beta</em>0} + \hat{\beta<em>1} x</em>{i1} + … + \hat{\beta<em>p} x</em>{ip}$

• Again, the “hat” notation is used.
  • $\hat{y_i}$ is the fitted value
  • $y_i$ is the actual observed value

Errors and Residuals

• Errors cannot be directly computed:

$\epsilon<em>i = y</em>i - \beta<em>0 - \beta</em>1 x<em>{i1} - … - \beta</em>p x_{ip}$

*b Since the coefficients $\beta<em>0, \beta</em>1, …, \beta_p$ are unknown.

• Errors are estimated by residuals:

$e<em>i = y</em>i - \hat{y<em>i} = y</em>i - (\hat{\beta<em>0} + \hat{\beta</em>1} x<em>{i1} + … + \hat{\beta</em>p} x_{ip})$

• $e<em>i ≈ \epsilon</em>i$ in general since $\hat{\beta<em>j} ≈ \beta</em>j$

Properties of Residuals

• The LS estimate $(\hat{\beta<em>0}, \hat{\beta</em>1}, …, \hat{\beta<em>p})$ satisfies the equations $\sum</em>{i=1}^{n}(y<em>i - \hat{\beta</em>0} - \hat{\beta<em>1} x</em>{i1} - … - \hat{\beta<em>p} x</em>{ip}) = \sum<em>{i=1}^{n} e</em>i = 0$
  and
  $\sum<em>{i=1}^{n} x</em>{ik}(y<em>i - \hat{\beta</em>0} - \hat{\beta<em>1} x</em>{i1} - … - \hat{\beta<em>p} x</em>{ip}) = 0, k = 1, 2, …, p$
• The residuals $e<em>i$ hence have the properties $\sum</em>{i=1}^{n} e<em>i = 0$, Residuals add up to 0 $\sum</em>{i=1}^{n} x<em>{ik}e</em>i = 0, k = 1, 2, …, p$
  Residuals are orthogonal to predictors.
• The two properties combined imply that the residuals have 0 correlation with each of the p predictors since
• $Cov(X<em>k, e) = \frac{1}{n-1} [\sum</em>{i=1}^{n} x<em>{ik}e</em>i - n\bar{x_k} \bar{e}] = 0$

Mean Square Error (MSE) — Estimate of $\sigma^2$

• The variance $\sigma^2$ of the errors $\epsilon_i$’s is estimated by the mean square error (MSE), the sum of squares of residuals divided by $n − p − 1$.

• $MSE = \frac{\sum<em>{i=1}^{n} e</em>i^2}{n - p - 1} = \frac{\sum<em>{i=1}^{n}(y</em>i - \hat{y_i})^2}{n - p - 1}$

• Why divided by $n − p − 1$ instead of by n?

• A simple reason is it takes at least $p + 1$ observations to estimate $\beta<em>0, \beta</em>1, …, \beta_p$.
  Need at least $p + 2$ observations to get non-zero residuals to determine the variability of the estimate

• We will show (in the next Lecture) that MSE is an unbiased estimator for $\sigma^2$.

Example: The Auto Data

• Auto data of 9 variables about 392 car models in the 1980s.
• The variables include
  • acceleration: Time to accelerate from 0 to 60 mph (in seconds)
  • horsepower: Engine horsepower
  • weight: Vehicle weight (lbs.)

How to Do Regression in R?

• R code:

lm(acceleration ~ weight + horsepower, data=Auto)

Output:

Call:
lm(formula = acceleration ~ weight + horsepower, data = Auto)

Coefficients:
(Intercept)       weight   horsepower
   18.4358     -0.0023      -0.0933

• The lm() command above asks R to fit the model $acceleration = \beta<em>0 + \beta</em>1 weight + \beta_2 horsepower + \epsilon$
• R gives us the regression equation

$acceleration = 18.4358 + 0.0023 weight − 0.0933 horsepower$

More R Commands

lm1 = lm(acceleration ~ weight + horsepower, data=Auto)
lm1$coef  # show the estimated beta's

Output:

(Intercept)       weight   horsepower
18.435791     -0.0023      -0.0933

lm1$fit # show the fitted values
lm1$res # show the residuals

plot(lm1$fit,lm1$res, xlab="Fitted Values", ylab="Residuals")

Interpretation of Regression Coefficients

Interpretation of the Intercept $\beta_0$

• $\beta_0$ = intercept = the mean value of Y when all Xj’s are 0.
  • May have no practical meaning
  • e.g., $\beta_0$ is meaningless in the Auto model as no car has 0 weight

Interpretation of the regression coefficient for $\beta_j$

• $\beta_j$ = the regression coefficient for Xj, is the mean change in the response Y when Xj is increased by one unit holding other Xi’s constant.

  • Also called the partial regression coefficients because they are adjusted for the other covariates
  • Interpretation of $\beta_j$ depends on the presence of other predictors in the model

• e.g., the 2 $\beta_1$’s in the 2 models below have different interpretations

  • Model 1 : $Y = \beta<em>0 + \beta</em>1 X<em>1 + \beta</em>2 X_2 + \epsilon$
  • Model 2 : $Y = \beta<em>0 + \beta</em>1 X_1 + \epsilon$

Something Wrong?

lm(acceleration ~ weight, data=Auto)$coef

Output:

(Intercept)       weight
19.572666     -0.001354

lm(acceleration ~ weight + horsepower, data=Auto)$coef

Output:

(Intercept)       weight   horsepower
18.435791     0.002302      -0.093313

• The coefficient $\hat{\beta_1}$ for weight is negative in the Model 1 but positive in the Model 2.
• Do heavier cars require more or less time to accelerate from 0 to 60 mph?

Effect of weight Not Controlling for Other Predictors

library(ggplot2)
ggplot(Auto, aes(x=weight, y=acceleration)) + geom_point()

• From the scatter plot above, are weight and acceleration are positively or negatively associated?
• Do heavier vehicles generally require more or less time to accelerate from 0 to 60 mph?
• Is that reasonable?

Effect of weight Controlling for horsepower (1)

ggplot(Auto, aes(x=weight, y=acceleration, col=horsepower)) + geom_point()
ggplot(Auto, aes(x=weight, y=acceleration, col=horsepower)) + geom_point() + scale_color_gradientn(colours = rainbow(5))

Effect of weight Controlling for horsepower (2)

• Consider car models of similar horsepower (similar color), are weight and acceleration positively or negatively correlated?

ggplot(Auto, aes(x=weight, y=acceleration, col=horsepower)) + geom_point() + scale_color_gradientn(colours = rainbow(5))

Effect of weight Controlling for horsepower (3)

R codes for the plot on the previous page

Auto$hp = cut(Auto$horsepower, breaks=c(45,70, 80, 90,100,110, 130, 150,230),
labels=c("hp<=70", "70 < hp <= 80", "80 < hp <= 90", "90 < hp <= 100", "100 < hp <= 110", "110 < hp <= 130", "130 < hp <= 150","hp> 150"))
ggplot(Auto, aes(x=weight, y=acceleration, col=horsepower)) + geom_point() + scale_color_gradientn(colours = rainbow(5)) + facet_wrap(~hp, nrow=2) + theme(legend.position="top")

#### Example: Auto Data — Simpson’s Paradox

• Why is the association between acceleration and weight flipped from positive to negative when horsepower is ignored?

• Heavier vehicles (purple dots) tend to have more horsepower while lighter ones (red dots) tend to have less

• Vehicles with more horsepower (purple dots) require less time to accelerate while those with less (red dots) require more

• Hence, when ignoring horsepower, it looks like heavier vehicles require less time to accelerate, though heavier vehicles require more time to accelerate after the effect of horsepower is adjusted (which means considering only vehicles with similar horsepower).

What We Mean by “Adjusted for Other Coveriates”?

• For a multiple linear regression model with p predictors

• $Y = \beta<em>0 + \beta</em>1 X<em>1 + · · · + \beta</em>p X_p + \epsilon$

• $\beta<em>j$ represents the effect of Xj on the response variable Y after it has been adjusted for all of $\begin{aligned}X</em>1, …, X_p \end{aligned}$
  except Xj.

What does “adjusted for” mean?

What We Mean by “Adjusted for Other Coveriates” (2)?

• The LS estimate $\hat{\beta<em>j}$ for $\beta</em>j$ in the MLR model $Y = \beta<em>0 + \beta</em>1 X<em>1 + · · · + \beta</em>p X_p + \epsilon$ would be identical to the slope for the SLR model computed as follows.
  1. Regress Y on all other Xk’s except Xj
  2. Regress Xj on all other Xk’s except Xj
  3. Fit a SLR model using the residuals from Step 1 as the response and the residuals from Step 2 as the predictor.
• Moreover, the intercept obtained in Step 3 would be 0.
• This proof of this result involves complicated matrix algebra and hence is omitted. We just illustrate with an example.

Example for the Auto Data

• recall we have fit the model

&amp;gt; acceleration = beta0 + beta1weight + beta2horsepower + $\epsilon$

• and obtained the estimate for $\beta$1 to be $\hat{\beta}$2 = 0.0023.
  1. Step 1. Regress acceleration on horsepower. Let RY be the residuals of this model.

```R
RY = lm(acceleration ~