312 Test 2

B^ OLS

(XtX)^-1Xty

H matrix

X(XtX)^-1Xt

H matrix properties

symmetric + idempotent

Var(B^) OLS

sigma²[(XtX)^-1]

TSS

yTy - ny-bar²

SSreg

yTHy - ny-bar²

SSE

yT(I - H)y

R² formula

1 - SSE/TSS

simga-hat²

SSE/(n-p)

t-stat

B^j / se(B^j)

f-stat

SSreg/(p-1) / SSE/(n-p)

MSE

SSE/n-p

B^ dist

B^ ~ N(B, sigma²(XTX)-1)

confidence interval for B^j

B^j ± ta/2,n-p * se(B^j)

confidence interval for E[ynew]

y^new ± ta/2,n-p * simga^ sqrt(xnewT(XTX)^-1xnew)

prediction interval for ynew

y^new ± ta/2,n-p * simga^ sqrt(1 + xnewT(XTX)^-1xnew)

R²adj

1 - (SSE/(n-p) / TSS/(n-1))

AIC

nlog(MSE) + 2p + C

Cp formula

SSE/(simga-hat²) - n +2p

LOOCV shortcut

1/n sum ( ( (yi - yi^)/(1-hi) )² )

leverage

hi = Hii; how much the pt influences own prediction

flag for potential outliers

if hi > 2p/n

Var(ei^)

simga² (1 - hi)

standardized residual

ei^ / simga^ sqrt(1-hi) (approx follows standard normal)

studentized residual

ei^ / simga^(i) sqrt(1-hi) (follows tn-p-1)

cook’s distance

Di = ri²/p * hi/1-hi

minimized quantity for ridge

(y-XB)T(y-XB) + lambdaBTB

B^ridge

(XTX+lambda I)^-1 XTy

E[B^ridge]

(XTX + lambda I)^-1 XTXB

Var[B^ridge]

sigma²(XTX + lambda I)^-1 XTX(XTX+lambda I)^-1

spline formula

y^ = Bo^ + B1^X + B2^X² + B3^X³ + Sum(B3+k^(X-knot)³) +where (x- knot)³+ = (x-knot)³ if x > knot else 0