Machine Learning and Prediction Models

Flashcards covering key machine learning concepts, statistical estimators, and model evaluation metrics based on lecture materials.

MSPE (Mean Squared Prediction Error)

The metric used to compare prediction models.

Unbiased Estimator

An estimator where $E(\hat{\mu}) - \mu = 0$.

Mean Squared Error of $\hat{\mu}$

The value is $1$ when the estimator has a sampling distribution of $N(0, 1)$ and the true population mean is $0$.

Mean Squared Error of $\tilde{\mu}$

The value is $0.5$ when the estimator has a sampling distribution of $N(0.5, (0.5)^2)$ and the true population mean is $0$.

$\bar{R}^2$ (Adjusted R-squared)

A metric that penalizes the inclusion of an additional explanatory variable if its associated $t$-statistic is less than $1$.

Supervised Learning

Machine learning that involves predicting an outcome with a set of explanatory variables.

Cross-validation

A method used for empirically tuning model complexity by dividing data into training and testing samples.

Training and Testing Samples

The initial divisions of data used to begin the cross-validation process.

Folds (Validation and Estimation)

The divisions of the training sample where one fold is held out for validation while the others are used to estimate the model.

MSPE Averaging

The cross-validation step that involves computing the Mean Squared Prediction Error for each fold and averaging them over all folds.

$\lambda$ (Lambda)

A value that determines the strength of the penalty imposed by the regularizer in cross-validation.

LASSO

A shrinkage estimator that performs variable selection by forcing the coefficients of the least relevant variables to be equal to $0$.