Citadel Quant Research Interview

Derive the OLS estimator for simple linear regression.

Derive the OLS estimator for multiple regression in linear algebra form.

What assumptions are needed for the OLS estimator expression to exist?

What does it mean for XTX to be singular? What could cause this? What does it mean when XTX is singular? How would you fit the model in this case?

Explain the geometric interpretation of the goals of linear regression.

Show why the OLS residuals are orthogonal to the columns of the data matrix X. How does the intercept of the model relate to the sum of the residuals?

Define the projection/hat matrix. Explain its important properties and its eigenvalues.

What assumptions are typically made about the error term in linear regression? Why are these assumptions made?

What assumptions are needed for the OLS estimator to be unbiased?

What assumptions are needed for the OLS estimator to be consistent?

What assumptions are needed for the OLS estimator to be efficient?

What assumptions are needed for the OLS estimator to be the Best Linear Unbiased Estimator?

What is heteroskedasticity? Does it make OLS biased?

What is autocorrelation of residuals?

What is multicollinearity?

Suppose you duplicate one column of X. What happens mathematically to OLS?

Explain ridge regression. Derive its closed-form estimator from.

Compare ridge and lasso regression. Why can lasso produce coefficients that are exactly zero while ridge generally cannot?