Machine Learning, Stats, Math

What is supervised learning?

Learning a mapping from inputs to outputs using labeled examples.

What is unsupervised learning?

Discovering structure in unlabeled data (clustering, dimesionality reduction)

Define overfitting and name two prevention methods.

Model fits noise; prevent with regularization, cross-validation, early stopping, or more data

Difference between classification and regression?

Classification predicts discrete labels; regression predicts continuous values.

How does k-NN predict class?

By majority vote among the k nearest neighbors in feature space.

Why scale features?

To ensure features are comparable, important for kNN, SVM and gradient descent

What is cross-validation?

Method to estimate model generalization by splitting data into training and validation folds.

What is the ROC curve?

Plot of True Positive Rate vs False Positive Rate; AUC summarizes discrimination ability.

What does regularization do?

Adds penalty to model weights to reduce overfitting; L1 (Lasso) and L2 (Ridge)

Define bias and variance.

Bias = error from incorrect assumptions; Variance = error from model sensitivity to data.

What is PCA?

Linear dimensionality reduction projecting data onto principal components with max variance.

What is SMOTE?

Oversampling technique that synthesizes new minority class samples to fix class imbalance.

What is the softmax function used for?

To convert logits to normalized class probabilities.

What is dropout?

Randomly setting neurons to zero during training to prevent overfitting.

What is early stopping?

Halting training when validation error stops decreasing to prevent overfitting.

What is ensemble learning?

Combining multiple models to improve performance, e.g. bagging or boosting

What is XGBoost known for?

Regularized gradient boosting, effective and efficient for tabular data

What is SHAP?

Shapley value-based feature attribution for interpretability.

What is feature drift vs concept drift?

Feature drift: change in X distribution; Concept drift: change in P(y|X).

Why calibrate models?

To make predicted probabilities match actual outcome frequencies (important in risk).

How does Random Forest reduce overfitting?

By averaging many decision trees trained on bootstrapped data.

Difference between bagging and boosting?

Bagging trains models in parallel to reduce variance; boosting trains sequentially to reduce bias.

What is t-SNE mainly used for?

Visualizing high-dimensional data in 2D while preserving local structure.

What is transfer learning?

Using pretrained models on new tasks to save time and data.

What problem does batch normalization solve?

Stabilizes layer activations, speeds convergence, and allows for higher learning rates

Name local and global model explanation methods.

Local: LIME; Global: SHAP or feature importance.

What is adversarial training?

Training with perturbed inputs to improve robustness.

What is CatBoost notable for?

Handles categorical variables natively without one-hot encoding.

What is out-of-bag (OOB) error?

Validation metric using unused samples per tree in a Random Forest.

What is model serialization?

Saving trained models for deployment, e.g. using pickle or joblib

What is quantile regression?

Estimates conditional quantiles (like median or 95th percentile).

What is feature hashing?

Hashes categorical features into fixed-length numeric vectors.

What is cross-validation leakage?

Information from validation leaks into training via preprocessing before splitting.

What is bootstrapping used for?

Resampling with replacement to estimate uncertainty and confidence intervals.

When to use precision vs recall?

Precision when false positives are costly; recall when false negatives are costly.

Difference between calibration and discrimination?

Calibration = probability accuracy; discrimination = ranking quality (AUC).

What are Bayesian neural networks used for?

Quantifying uncertainty in predictions, useful for risk-sensitive tasks.

What is knowledge distillation?

Training a smaller model to mimic the outputs of a larger model.

Define concept drift detection best practice.

Monitor feature distributions and predictive performance; retrain when drift detected.

What is active learning?

Model selects most informative samples to query for labels, reducing labelling cost

Formula for sample variance?

s² = (1/(n-1)) sigma (xi - xbar)².

Bayes theorem?

P(A|B) = [P(B|A) * P(A)] / P(B).

Central Limit Theorem (CLT)?

Sample mean of i.i.d. variables approximates a Normal distribution as n approaches infinity

When to use t-test vs z-test?

Use t-test when σ unknown or n small; z-test when σ known or n large.

Use-case for Poisson distribution?

Modeling counts of rare events per time unit (e.g. fraud events)

Law of total probability?

P(B) = sigmai P(B|Ai)P(Ai) for a partition of Ai.

Meaning of p-value?

Probability of observing data at least as extreme assuming H0 is true.

MLE vs Method of Moments?

MLE maximizes likelihood; MoM matches sample and population moments.

When to use Poisson vs Binomial?

Poisson for rare events; Binomial for fixed number of independent trials.

Complexity of BFS?

O(V + E).

Binomial coefficient formula?

C(n, k) = n! / [k!(n-k)!]

Use-case for Dijkstra?

Finding shortest path with non-negative edge weights.

What is dynamic programming?

Breaking problems into overlapping subproblems and caching results.

Master theorem concept?

Compare f(n) vs n^(log_b a) to estimate asymptotic complexity.

Generating function use?

Solve recurrences and count combinatorial objects.

What is modular arithmetic used for?

Cryptography and hashing operations.

Definition of NP-hard?

As hard as hardest problems in NP, not necessarily solvable in polynomial time

Recurrence relation for Fibonacci?

Fn = Fn-1 + Fn-2, F0 = 0, F1 = 1

Example of greedy optimal algorithm?

Kruskals algorithm for Minimum Spanning Tree.