Machine Learning Principles

Unsupervised Learning

Model learns patterns from unlabelled data without explicit teaching signals (clustering)

Supervised learning

Learning from labelled data to predict outputs from inputs (regression, classification)

Regression

Supervised learning with continuous output

Classification

Supervised learning with discrete categories as output

Underfitting

Model is too simple to capture patterns in training data

Overfitting

Model is too complex, fitting noise in the training data

Model selection

Process of choosing the best model/hyperparameters for a given dataset, often using cross validation or validation data

Training dataset

Data used to fit model parameters

Validation dataset

Dataset used to tune hyperparameters and select the best model

Test dataset

separate dataset used to evaluate final model performance

Cross validation

Evaluates model performance by splitting data into multiple folds, training on some and validating on others

No free lunch theorem

No universally best model

Model parameters

Values learned by the model from the training data

Parametric model

Model defined by a fixed number of parameters, independent of dataset sizeN

Nonparametric mdoel

Model whose complexity can grow with the amount of data

Likelihood function

Function expressing the probability of observingt he data given a set of parameters

MLE

Estimates model parameters by maximising the likelihood function