9 Regression forecasting

What is the multiple linear-regression model?

y_i = β0 + ∑_j β_j*x_ij + ε_i. It models a numerical response, and a coefficient describes a conditional relationship holding the other included predictors fixed.

Does linear regression require the observed response values to be normally distributed?

No. Normal-error assumptions concern particular inferential procedures under a model, not a requirement that the marginal y histogram itself be normal.

In what sense is a regression model linear?

It is linear in its coefficients. Including a chosen transformed predictor, such as x², can still produce a coefficient-linear regression model.

What is the R-squared formula?

R² = 1 - SSE/SST. SSE is residual sum of squares and SST total sum of squares around the response mean, for the usual intercept-based in-sample setting.

What is the adjusted R-squared formula?

Adjusted R² = 1 - (1-R²)*(n-1)/(n-p-1), where p is the number of predictors excluding the intercept and n > p+1.

Why can adding a predictor increase R-squared but decrease adjusted R-squared?

For nested least-squares fits on the same data, extra predictors cannot increase the minimum training SSE. Adjusted R² also penalizes predictor count, so a tiny fit gain may not offset the added complexity.

Do high, similar R-squared and adjusted R-squared prove future predictive accuracy?

No. They describe in-sample fit and a predictor-count adjustment. They do not rule out leakage, temporal changes, overfitting or poor future performance.

What is the simple moving-average forecasting formula?

F_t = (1/m)*∑(j=1 to m) y(t-j). It predicts time t from the previous m actual observations and excludes the unknown y_t.

How does a larger moving-average window affect a forecast?

It generally smooths more and reacts more slowly to change. A shorter window is more responsive but does not always have smaller future error.

Example: observations are 100, 110, 90 and 120. What is the next 3-observation SMA forecast?

(110 + 90 + 120)/3 = 106.67. The first observation, 100, is outside the latest three-value window.

What is the simple exponential-smoothing update?

F_(t+1) = αy_t + (1-α)F_t. Combine the newest actual observation with the previous forecast, using a chosen smoothing constant.

How does increasing smoothing alpha change responsiveness?

It puts more weight on the newest observation and less on the previous forecast. Greater responsiveness does not guarantee smaller future error.

Why are exponential-smoothing weights called exponential?

Successive past-observation weights are α, α(1-α), α(1-α)² and so on. Earlier information retains geometrically decaying influence.

What happens at smoothing alpha equal to 1 or 0?

α = 1 gives the last actual value as the next forecast, the naive forecast. α = 0 never updates the initialized forecast.

Example: previous forecast = 100, actual = 120, alpha = 0.25. What is the next forecast?

0.25120 + 0.75100 = 105.

Why should initialization errors be distinguished from genuine forecast errors?

If the initialized forecast is set equal to an observed value, its zero error is imposed by initialization rather than earned by predicting an unknown outcome.

How do rolling one-step and fixed-origin multi-step forecasts use information differently?

Rolling forecasts update after each actual value becomes available. A fixed-origin multi-step forecast cannot use future realized values from beyond its origin.

How is a signed prediction error defined in the guide?

e_i = y_i - yhat_i. Positive means underprediction and negative means overprediction under this sign convention.

What is the mean squared error formula?

MSE = (1/n)*∑_i (y_i-yhat_i)². Squaring prevents sign cancellation and gives large errors disproportionate influence.

What is the root mean squared error formula?

RMSE = sqrt(MSE) = sqrt[(1/n)*∑_i (y_i-yhat_i)²]. It returns the error scale to the response units.

What is the mean absolute error formula?

MAE = (1/n)*∑_i |y_i-yhat_i|. In forecasting, the guide also calls this mean absolute deviation, MAD.

What is the mean absolute percentage error formula?

MAPE = (100/n)*∑_i |(y_i-yhat_i)/y_i|, for nonzero actual values. It is undefined at zero and unstable near zero.

Example: signed errors are 2 and -4. Find MAE, MSE and RMSE.

MAE = (2+4)/2 = 3. MSE = (4+16)/2 = 10. RMSE = sqrt(10) ≈ 3.16.

Why is mean signed error not interchangeable with MAE?

Positive and negative errors can cancel. A low signed average can conceal large errors in opposite directions.

Why can MAD be ambiguous?

Forecasting MAD often means mean absolute prediction error. In robust statistics it commonly means median absolute deviation from the median, a different quantity.

What is unweighted KNN regression’s prediction formula?

yhat(x) = (1/K)*∑_(i in N_K(x)) y_i. Average the numerical responses of the K nearest training neighbors; N_K(x) is the chosen neighbor set.

Example: the three nearest training responses are 10, 14 and 18. What does KNN regression predict?

(10 + 14 + 18)/3 = 14. Classification would instead combine categorical labels through votes.