Statistical Learning - Chapters 2, 3, and 4

Flashcards based on lecture notes from 'An Introduction to Statistical Learning', Chapters 2, 3, and 4, focusing on key concepts and methods.

What is Statistical Learning?

Methods for understanding relationships between variables and making predictions.

What is Supervised Learning?

Learning with both inputs and corresponding outputs, divided into regression and classification.

What is Unsupervised Learning?

Learning with only inputs, used for clustering and dimension reduction.

What is Prediction in Statistical Learning?

The goal is to accurately forecast future values.

What is Inference in Statistical Learning?

Understanding the relationships between predictors and responses.

What are Parametric methods?

Methods that assume a specific functional form, like linear regression.

What are Non-parametric methods?

Methods that do not assume a specific functional form; very flexible.

What is Mean Squared Error (MSE)?

Used to measure accuracy in regression problems.

What is the Bias-Variance Trade-off?

Balancing model complexity to avoid underfitting (high bias) and overfitting (high variance).

What are Simple Models?

They have high interpretability but risk high bias.

What are Complex Models?

They reduce bias but increase variance, possibly causing overfitting.

What is Linear Regression?

A supervised learning technique used for predicting quantitative outcomes, assuming a linear relationship between predictors and response.

What is the Residual Sum of Squares (RSS)?

Coefficient estimation through least squares minimizes?

What is Multiple Linear Regression?

Extends simple regression by considering multiple predictors.

What is Residual Standard Error (RSE)?

Measures prediction error magnitude.

What is the R² Statistic?

Proportion of variance in the response explained by predictors.

What are Qualitative Predictors?

Categorical variables represented through dummy variables.

What are Interaction Terms?

Capture the interaction between predictors.

What are Nonlinear Relationships in Linear Regression?

Polynomial or transformation-based adjustments to predictors.

What does classification involve?

Predicting categorical responses.

What is Logistic Regression?

Estimates probabilities using the logistic (sigmoid) function.

What is Maximum Likelihood?

Coefficients in logistic regression are estimated via?

What is Linear Discriminant Analysis (LDA)?

Assumes classes share a common covariance matrix.

What is Quadratic Discriminant Analysis (QDA)?

Extends LDA by allowing each class its covariance matrix.

What is K-Nearest Neighbors (KNN)?

Classifies by majority vote of nearest neighbors.

What is a Confusion Matrix?

Detailed breakdown of correct and incorrect classifications.

What are common evaluation metrics in classification?

Accuracy, sensitivity, specificity, and ROC curves.