Machine Learning Practice Flashcards

This set of flashcards covers fundamental machine learning concepts, algorithms, evaluation metrics, and advanced topics extracted from the provided university lecture notes.

Machine Learning

A field where computers learn from data and improve their performance over time.

Machine Learning (Tom Mitchell, 1997)

Improving performance on a task T based on experience E.

Supervised Learning

A type of machine learning that requires labeled data for training.

Unsupervised Learning

A type of machine learning used to find patterns or group data without pre-defined labels.

Reinforcement Learning

Learning through rewards and punishments while interacting with an environment.

Deep Learning

A subfield of Machine Learning involving neural networks with many layers.

Feature Engineering

The process of creating, selecting, and transforming raw data into suitable features for a model.

Training Set

The dataset used to train a machine learning model.

Validation Set

The dataset used to optimize hyperparameters during the training process.

Testing Set

The dataset used for objective evaluation of a model's performance after training.

Hyperparameters

Parameters set by the user before training to control the learning process, such as the learning rate.

Parameters

Internal values, like weights in a neural network, that the model learns from the data.

Linear Regression

A model that represents the output Y as a linear function of one or more input features.

Gradient Descent

An optimization algorithm used to minimize the loss function by iteratively updating model weights.

RMSE (Root Mean Squared Error)

A measure of the average magnitude of error, calculated as the square root of the average of squared differences between prediction and truth.

MAE (Mean Absolute Error)

The average of the absolute differences between the actual values and the predicted values.

Overfitting

A situation where a model fits the training data too closely but fails to generalize to new data.

Underfitting

A situation where a model is too simple to learn the underlying relationships in the data.

Cross-validation

A technique used to estimate a model's true error and help select the best model by partitioning the data.

K-Nearest Neighbors (KNN)

A non-parametric, lazy learning algorithm that classifies data based on the majority vote of its K closest neighbors.

Logistic Regression

A classification algorithm that uses a sigmoid function to predict the probability of a binary or multi-class outcome.

Sigmoid Function

An activation function that maps any real-valued number into a range between $0$ and $1$.

Accuracy

A performance metric calculated as the ratio of correct predictions to the total number of samples.

Precision

The ratio of correctly predicted positive observations to the total predicted positives (TP / (TP + FP)).

Recall (Sensitivity)

The ratio of correctly predicted positive observations to all actual positives (TP / (TP + FN)).

F1-score

The harmonic mean of precision and recall.

ROC Curve

A graph showing the relationship between the True Positive Rate (TPR) and the False Positive Rate (FPR).

Support Vector Machine (SVM)

A supervised learning model that finds the hyperplane that maximizes the margin between two classes.

Kernel Trick

A technique used in SVM to handle non-linear data by performing calculations in a high-dimensional space without explicitly mapping the data.

Naive Bayes

A classification algorithm based on Bayes' theorem with the strong assumption of conditional independence between features.

Perceptron

The simplest type of artificial neural network used for linear classification.

Backpropagation

A fundamental algorithm for training neural networks based on the chain rule to calculate gradients of the loss function.

ReLU (Rectified Linear Unit)

An activation function defined as $f(x) = \max(0, x)$, which helps reduce the vanishing gradient problem.

Convolutional Neural Network (CNN)

A type of deep neural network specialized for processing data with a grid-like topology, such as images.

Recurrent Neural Network (RNN)

A type of neural network designed for sequential data, where outputs from previous steps are used as inputs for current steps.

Clustering

The process of grouping a set of objects in such a way that objects in the same group are more similar to each other than to those in other groups.

K-means

A centroid-based clustering algorithm that partitions data into K clusters by minimizing the distance between points and their group's centroid.

DBSCAN

A density-based clustering algorithm that can find clusters of arbitrary shapes and identify outliers as noise.

PCA (Principal Component Analysis)

A dimensionality reduction technique that transforms data into a new coordinate system using eigenvectors and eigenvalues.

Generative Adversarial Network (GAN)

A deep learning architecture consisting of a Generator and a Discriminator that compete against each other to create synthetic data.