Chapter 7 - Theorems/ideas - Matrix Algebra

Symmetric Matrices

A matrix A is symmetric if:

• A^T = A

• Must be square

• Entries mirror across the diagonal

If you see A^T = A → strong eigenvalue guarantees

Theorem 1 - Orthogonality of Eigenvectors

If A is symmetric, then eigenvectors corresponding to distinct eigenvalues are orthogonal

Guarantees:

• No Gram-Schmidt needed between different eigenspaces

Fast construction of orthonormal eigenbasis

Orthogonal Diagonalization

A matrix A is orthogonally diagonalizable if

• A = PDP^T

where P is orthogonal and D is diagonal

Meaning:

• P^-1 = P^T

• Columns of P are orthonormal eigenvectors

Simplifies powers and quadratic forms

Symmetric - Orthogonally Diagonalizable

A matrix A is orthogonally diagonalizable if and only if A is symmetric

This is a two way equivalence

“Can I diagonalize with an orthogonal matrix?” - check symmetry

Theorem 3 - Spectral Theorem

If A is symmetric, then:

• All eigenvalues are real

• Geometric multiplicity = algebraic multiplicity

• Eigenspaces are mutually orthogonal

• A is orthogonally diagonalizable

This theorem ends all diagonalization questions

Spectral Decomposition

If A = PDP^T with orthonormal eigenvectors u_i :

• A = picture

Each term is rank - 1

Each u_iu_i^T is a projection matrix

Shows how A acts direction by direction

Quadratic Form

A quadratic form on Rⁿ is a function:

• Q(x) = x^TAx

where A is symmetric

Off diagonal entries create cross terms x_ix_j

Change of Variable in Quadratic Forms

If x = Py, then

• x^TAx = y^T(P^TAP)y

Guarantees:

• New matrix is P^TAP

Used to remove cross terms

Theorem 4 - Principle Axes Theorem

If A is symmetric, there exists an orthogonal change of variables that transforms

• x^TAx into y^TDy

with no cross-product terms (so no distorted axes, they’ll be orthogonal)

• Quadratic form becomes diagonal

This is guaranteed for symmetric matrices

Classification of Quadratic Forms

A quadratic form Q(x) = x^TAx is:

This determines shape of level curves

Theorem 5 - Quadratic Forms & Eigenvalues

If A is symmetric:

No completing the square needed