DATS2103 Quiz 5

Why do we move beyond linearity?

Because real-world relationships are often nonlinear, and linear models can produce high error

What are basis funcitons?

Transformations of predictors (like polynomials, steps, splines) to make linear models more flexible

What does polynomial regression do?

Adds curved terms (e.g., X²,X³) to capture nonlinear relationships

Main wekness of polynomial regression?

Can overfit and behave poorly at the boundaries

What is a step funciton (piecewise-constant regression)?

Splits a variable into bins (like age groups) and fits constant values in each bin.

Weakness of step functions?

Discontinuous at breakpoints and poor at boundaries

What are regressin splines?

Piecewise polynomials joined at knots for smooth curves.

What are knots?

Points where the funciton changes behavior but remains  smooth

How are natural splines different from regression splines?

they force the funciton to be linear at the boundaries to prevent extreme swings

How do smoothing splines control flexibility?

Use a penalty/tuning paramter (λ) to balance smoothness vs flexibility.

What happens with small vs large (λ)?

Small → very wiggly curve; large → smoother, straighter curve

What is local regression (LOESS/LOWESS)?

Fits regression in small neighborhoods of the data

What controls smoothness in local regression?

The span (proportion of nearby points used).

• Small span → very flexible/wiggly

• Large span → smoother/less flexible

What do GAMs allow?

Nonlinear functions for each predictor, while keeping additiveity

Why are GAMs useful?

They handle multiple predictor flexibly and allow easy interpretaiton of each varibale’s effect

Which method is simplest but risky with edges?

Polynomial regression

Which method is good for natural categories but discontinuous?

Step functions

Which mehtod balances flexibility and smoothness with knots?

Regression splines

Which method uses a penalty to control smoothness?

Smoothing splines

Which method is like kNN but uses regresion?

Local regression

Which method combines everything for multiple predictors?

Generalized Addtive Models (GAMs)

Which mehtod avoids wild swings at the boundaries?

Natural splines