Singular Value Decomposition (SVD) Notes

Flashcards summarizing key vocabulary and concepts from the Singular Value Decomposition lecture notes.

Singular Value Decomposition (SVD)

A method to express a matrix as a product of three matrices: U, Σ, and V^T, containing singular values and orthogonal matrices.

Left singular vector matrix (U)

An orthogonal matrix in SVD representing left singular vectors.

Right singular vector matrix (V)

An orthogonal matrix in SVD representing right singular vectors.

Singular values (σj)

Non-negative values in the diagonal matrix Σ representing the importance of corresponding singular vectors.

Economy-size SVD

A compact SVD that uses fewer vectors corresponding to the non-zero singular values.

Truncated SVD

An approximation of SVD retaining only the largest singular values and their corresponding vectors.

Column space

The space spanned by the columns of a matrix.

Rank of a matrix

The dimension of the column space or the row space of the matrix, equal to the number of non-zero singular values.

Identity matrix (In)

A square matrix with ones on the diagonal and zeros elsewhere; has all singular values equal to one.

Rank of the zero matrix

The rank of a zero matrix is 0.

Distance to singularity

A measure of how close a matrix is to being singular, characterized by the smallest singular value.

Orthogonal matrix

A square matrix whose rows and columns are orthogonal unit vectors; its inverse is its transpose.

PCA (Principal Component Analysis)

A statistical procedure that uses SVD for dimensionality reduction by transforming to a new set of variables (principal components).

Gram matrix

A matrix of inner products that can be used to find the sample covariance from centered data.

Sample variance

A measure of the spread of data points in a sample, calculated as the average of the squared deviations from the mean.

Covariance matrix (C)

A matrix that contains covariances between pairs of variables in a dataset.