Dimensionality Reduction in Machine Learning

A set of vocabulary flashcards focusing on terms and concepts related to dimensionality reduction techniques used in machine learning.

Dimensionality Reduction

The process of reducing the number of features under consideration.

Feature Selection

The technique of selecting a subset of relevant features for use in model construction.

Feature Extraction

The technique of transforming data from a high-dimensional space to a lower-dimensional space.

Principal Component Analysis (PCA)

An unsupervised technique used for dimensionality reduction by projecting data in the direction of largest variance.

Linear Discriminant Analysis (LDA)

A supervised technique that maximizes class separability.

Covariance Matrix

A matrix that provides a measure of the directions along which two or more random variables vary together.

Eigenvalues

Scalar values that indicate the magnitude of variance in a dataset along an eigenvector.

Eigenvectors

Non-zero vectors that change at most by a scalar factor when a linear transformation is applied.

Kernel PCA

A non-linear dimensionality reduction technique that uses kernel methods to project data into higher-dimensional spaces.

Variance

A measure of the dispersion of a set of data points.

Information Loss

The loss of significant information that occurs when transforming data into a lower-dimensional space.

Projection Matrix

A matrix used to project data points into a different feature space.