AI Written + Maths Questions

Explain the difference between underfitting, good fit, and overfitting, using the polynomial model example from sensor calibration.

Underfitting: Model too simple, cannot capture trend; both training and test errors are high, e.g. fitting a straight line to a clearly curved sensor response.

Good fit: Model has appropriate complexity; low training and test errors; captures underlying relationship without following noise.

Overfitting: Model too complex, e.g. a 10th-order polynomial passing exactly through 100 calibration points; training error very low but test/interval behaviour unstable and poor.

Describe L2 (Ridge) and L1 (LASSO) regularization and how they affect model weights and feature usage.

L2 regularization adds a penalty proportional to the sum of squared weights, discouraging large weights so that all features tend to have small but non-zero influence.

L1 regularization adds a penalty proportional to the sum of absolute weights, encouraging some weights to become exactly zero, which effectively performs feature selection.

Both methods trade off prediction error and model simplicity, controlled by a regularization parameter.

Compare supervised and unsupervised learning in terms of data requirements, typical tasks, and example algorithms discussed in the lectures

Supervised learning uses labelled data (input–output pairs), typical tasks include regression and classification, with algorithms such as linear regression, SVM, k-NN, and neural networks.

Unsupervised learning uses unlabelled data to discover structure; typical tasks include clustering and dimensionality reduction; algorithms include k-means, hierarchical clustering, PCA, autoencoders.

Supervised aims to predict outputs; unsupervised aims to find patterns, groups, or compressed representations.

Describe the main components of the LSTM cell (cell state and three gates) and explain how they help with vanishing and exploding gradients

Cell state Ct acts as long-term memory, allowing information to flow with minimal modification across time steps, forming a “gradient highway”.

Forget gate decides what information from the previous cell state to erase; input gate decides what new information to add; output gate controls what part of the cell state is exposed as hidden state.

Gated, mostly element-wise operations keep derivatives close to 1 and avoid repeated full matrix multiplications along the memory path, greatly reducing vanishing and exploding gradients.

Outline K-fold cross-validation: how it works and why it provides a better estimate of performance than a single train-test split

Procedure: Split data into K folds; for each fold, train on the other K−1 folds and test on the held-out fold; average performance across all folds.

Each point is in a test set exactly once, reducing dependence on one particular split and providing a more reliable estimate of generalization.

Useful for hyperparameter selection (e.g. model complexity, regularization strength) and model comparison.

A linear model for sensor output y (in volts) as a function of input x is

𝑦 = (𝑤0) + (𝑤1)𝑥

with learned parameters 𝑤0 = 0.5, 𝑤1 = 2.0. Compute:

a) Predicted output for x = 1.

b) Predicted output for x = 3

a) y = 0.5 + 2.0×1 = 2.5 V.

b) y = 0.5 + 2.0×3 = 6.5 V.

For N1, the actual measurements are 𝑦𝑡𝑟𝑢𝑒 1 = 3.0V and 𝑦𝑡𝑟𝑢𝑒 3 = 6.0V. Compute the Mean Squared Error (MSE) over these two data points.