Model Selection

Process of simplifying models by removing unnecessary variables

Variable Pruning

Model resulting from variable pruning which is more concise and effective

Parsimonious Model

Model which includes all available explanatory variables

Full Model

A metric indicating model fit strength and value as a predictor

Adjusted R-squared

Strategy which involves adding/deleting one variable at a time for model selection

Step-wise Selection

Starting with the full model, then removing one variable at a time based on improvement in the adjusted r-squared

Backwards Elimination with Adjusted R-squared

Starting with no predictors, and adding one at a time based on importance to the adjusted r-squared

Forwards Selection with Adjusted R-squared

Starting with the full model, then removing predictors based on the largest p-value to the point of the significance level

Backwards Elimination with P-value

Starting with no predictors, and adding one at a time based on the smallest p-value to the point of the significance level

Forwards Selection with P-value

Adjusted R-squared should be used when accuracy is most important; P-value should be used when we want to understand which variables are statistically significant predictors or produce a simpler model.

Adjusted R-squared Versus P-value