Statistical Learning - TDS10 Data Analysis for Business

Practice questions covering the introductory concepts of statistical learning, including problem settings, software, estimation goals (prediction vs. inference), and estimation approaches (parametric vs. nonparametric).

What is the course code and title provided in the transcript?

TDS10 - Data Analysis for Business

What is R, as described in the software section?

A programming language based on C designed for statistical analysis; it is open source and available via CRAN.

What is RStudio and what are its requirements?

A graphical user interface (GUI) that requires R to be already installed; it is used to run R, make plots, and create documents.

List three specific examples of statistical learning problems mentioned in the notes.

1. Identify risk factors for prostate cancer. 2. Predict heart attacks based on demographic, diet, and clinical measurements. 3. Customize an email spam detection system.

In the spam detection example, what was the data source and the input features?

The source was 4,601 emails sent to George at HP labs before 2000; the input features were the relative frequencies of 57 commonly occurring words and punctuation marks.

What are the two main learning approaches distinguished in the notes?

Supervised learning and Unsupervised learning.

What are the four components of every statistical learning problem?

1. Input variable(s) ($x$: predictors or features). 2. Output variable ($y$: outcome or response). 3. Data ($D$: set of observed variables). 4. Model ($f$).

What is the definition of a random variable according to the lecture?

A mathematical object that maps random outcomes into numbers (i.e., a function) equipped with a probability distribution of outcome occurrence.

How does the transcript distinguish between probability distributions for categorical and quantitative variables?

Mass ($\text{pdf}$) for categorical (discrete) and density ($\text{pmf}$) for quantitative (continuous).

What are the two main reasons for estimating the unknown function $f$?

1. Prediction. 2. Inference.

Compare the 'Prediction' and 'Inference' goals using the transcript's terminology.

Prediction is a 'black-box' approach focused on accurate predictions for $y$ regardless of the form of $f$, while Inference focuses on understanding the association between $y$ and $x$.

What are the two components into which prediction error $y - \tilde{y}$ can be decomposed?

Reducible error (can be improved by estimation technique) and Irreducible error (lower bound due to the distribution of possible $y$ values).

What is the mathematical decomposition of expected prediction error?

E[(y - \tilde{y})^2] = [f(x) - \tilde{f}(x)]^2 + \text{Var}(\text{̑})

How is the distance $d(y, \tilde{y})$ defined for quantitative and qualitative variables?

For quantitative: $d(y, \tilde{y}) = (y - \tilde{y})^2$. For qualitative: $d(y, \tilde{y}) = I(y \neq \tilde{y})$.

What is a training set?

A set of $n$ data points used to train, or teach, a statistical method how to estimate the unknown function $f$.

What is the difference between parametric and nonparametric estimation approaches?

Parametric approaches assume a simple functional form for $f$, whereas nonparametric approaches do not assume any functional form.

What are the two steps of the parametric model-based approach?

i) Assume a functional form (e.g., linear) and identify unknown parameters/coefficients. ii) Choose a procedure (e.g., OLS) to fit/train the model so that the result well approximates the outcome.

What is the primary trade-off mentioned for both parametric and nonparametric methods?

The trade-off between underfitting (simplicity/interpretability/smoothness) and overfitting (flexibility/accuracy/wiggliness).

What are the requirements and advantages of nonparametric methods?

They offer higher accuracy than parametric methods but require large datasets (far more than parametric approaches) and avoid explicit assumptions on functional form.