Diagonalization of Matrices

Diagonalization Theory: Part 1 and Part 2

The Diagonalization Theorem (Part 1)

• Statement: If a square matrix $A$ has an eigenvector basis, then $A$ can be expressed as a special product of matrices: $A = PDP^{-1}$.

  • $P$ is a matrix whose columns are the eigenvectors of $A$. These eigenvectors form the eigenvector basis.

  • $D$ is a diagonal matrix. Its diagonal entries are the eigenvalues of $A$. The eigenvalues are repeated according to their algebraic multiplicity.

  • Correspondence: The $i$-th column of $P$ (an eigenvector) must correspond to the $i$-th diagonal entry of $D$ (its respective eigenvalue).

• Benefit: This factorization simplifies calculations involving powers of $A$.

  • For example, $A^k = (PDP^{-1})^k = PD P^{-1} P D P^{-1} … P D P^{-1} = P D^k P^{-1}$ (where intermediate $P^{-1}P$ terms cancel out).

  • Calculating $D^k$ for a diagonal matrix $D$ is straightforward: if $D = \text{diag}(\lambda<em>1, \lambda</em>2, …, \lambda<em>n)$, then $D^k = \text{diag}(\lambda</em>1^k, \lambda<em>2^k, …, \lambda</em>n^k)$.

• Example 1: Triangular Matrix

  • Given matrix $A = \begin{pmatrix} 2 &amp; 0 &amp; 1 \ 0 &amp; 2 &amp; 2 \ 0 &amp; 0 &amp; 3 \end{pmatrix}$.

  • Eigenvalues: Since $A$ is a triangular matrix, its eigenvalues are the diagonal entries: $\lambda<em>1 = 2$, $\lambda</em>2 = 2$, $\lambda_3 = 3$.

  • Eigenvectors: (Provided) $v<em>1 = \begin{pmatrix} 1 \ 0 \ 0 \end{pmatrix}$, $v</em>2 = \begin{pmatrix} 0 \ 1 \ 0 \end{pmatrix}$, $v_3 = \begin{pmatrix} 1 \ 2 \ 1 \end{pmatrix}$.

  • Factorization: $A = P D P^{-1}$

    • $P = \begin{pmatrix} 1 &amp; 0 &amp; 1 \ 0 &amp; 1 &amp; 2 \ 0 &amp; 0 &amp; 1 \end{pmatrix}$ (columns are eigenvectors).

    • $D = \begin{pmatrix} 2 &amp; 0 &amp; 0 \ 0 &amp; 2 &amp; 0 \ 0 &amp; 0 &amp; 3 \end{pmatrix}$ (diagonal entries are corresponding eigenvalues).

    • $P^{-1}$ must be calculated (not immediately obvious).

• Example 2: Markov Chain Transition Matrix

  • Eigenvectors: $v<em>1 = \begin{pmatrix} 3 \ 5 \end{pmatrix}$ (for $\lambda</em>1 = 1$) and $v<em>2 = \begin{pmatrix} 1 \ -1 \end{pmatrix}$ (for $\lambda</em>2 = 0.92$).

  • Factorization: $T = \begin{pmatrix} 3 &amp; 1 \ 5 &amp; -1 \end{pmatrix} \begin{pmatrix} 1 &amp; 0 \ 0 &amp; 0.92 \end{pmatrix} P^{-1}$. The matrix $P^{-1}$ would need to be calculated explicitly.

The Diagonalization Theorem (Part 2)

• Statement: Let $A$ be an $n \times n$ matrix. If $A$ can be expressed as $A = PDP^{-1}$ where $P$ is an invertible matrix and $D$ is a diagonal matrix, then:

  • The columns of $P$ form an eigenvector basis for $A$.

  • The corresponding eigenvalues are the diagonal entries of $D$.

• Significance: This allows us to construct a matrix $A$ with pre-chosen eigenvalues and eigenvectors. It is the reverse of Part 1.

• Example: Constructing a Matrix

  • Choose eigenvectors: $v<em>1 = \begin{pmatrix} 1 \ 4 \end{pmatrix}$ and $v</em>2 = \begin{pmatrix} 3 \ 8 \end{pmatrix}$. (These must be linearly independent to form a basis for $R^2$).

  • Choose eigenvalues: $\lambda<em>1 = \pi^e$ and $\lambda</em>2 = -10^{10^{10}}$

  • Construct $P = \begin{pmatrix} 1 &amp; 3 \ 4 &amp; 8 \end{pmatrix}$ and $D = \begin{pmatrix} \pi^e &amp; 0 \ 0 &amp; -10^{10^{10}} \end{pmatrix}$.

  • The matrix $A = PDP^{-1}$ will then have these exact chosen eigenvalues and eigenvectors.

Core Concepts and Terminology

• Diagonalizable Matrix: A square matrix $A$ is called diagonalizable if it can be expressed in the form $A = PDP^{-1}$ where $P$ is invertible and $D$ is diagonal.

• Equivalence: A square matrix $A$ is diagonalizable if and only if there exists an eigenvector basis with respect to $A$. This means the existence of $A=PDP^{-1}$ and the existence of an eigenvector basis are two sides of the same coin.

• Rephrased Equivalence: An $n \times n$ matrix is diagonalizable if and only if there exist $n$ linearly independent eigenvectors of that matrix.

  • Note: All eigenvectors must have $n$ entries (i.e., be vectors in $R^n$) for the matrix multiplication to be valid.

• Diagonalization of A: The process of writing $A$ in the form $PDP^{-1}$.

• Diagonalization vs. Row Reduction: These are not the same. Row reduction typically changes the eigenvalues of a matrix, while diagonalization preserves them while transforming the matrix into a simpler form via its eigenvectors.

Determining if a Matrix is Diagonalizable

Case 1: Matrix with Distinct Eigenvalues

• Principle: Eigenvectors corresponding to distinct eigenvalues are linearly independent.

• Theorem: If an $n \times n$ matrix has $n$ distinct eigenvalues, then it is diagonalizable.

• Example: $A = \begin{pmatrix} -2 &amp; 1 \ 0 &amp; 1 \end{pmatrix}$.

  • Eigenvalues: $\lambda<em>1 = -2$, $\lambda</em>2 = 1$ (from diagonal of triangular matrix).

  • Since $A$ is a $2 \times 2$ matrix and has two distinct eigenvalues, it is immediately diagonalizable. We don't need to find the eigenvectors explicitly to know this.

• Example: A $3 \times 3$ triangular matrix with eigenvalues $0, 3, 5$ is diagonalizable because it has $3$ distinct eigenvalues.

Case 2: Matrix with Repeated Eigenvalues

• If a matrix does not have $n$ distinct eigenvalues, it might still be diagonalizable, but further investigation is required.

• Example (Not Diagonalizable): $B = \begin{pmatrix} 1 &amp; 1 \ 0 &amp; 1 \end{pmatrix}$.

  • Eigenvalue: $\lambda_1 = 1$ (from diagonal of triangular matrix). Its algebraic multiplicity is $2$ (since it's a $2 \times 2$ matrix).

  • Finding Eigenvectors: For $\lambda<em>1 = 1$, we solve $(B - 1I)x = 0$. This is $\begin{pmatrix} 0 &amp; 1 \ 0 &amp; 0 \end{pmatrix} \begin{pmatrix} x</em>1 \ x_2 \end{pmatrix} = \begin{pmatrix} 0 \ 0 \end{pmatrix}$.

    • This implies $x<em>2 = 0$ and $x</em>1$ is a free variable.

    • The eigenvectors are of the form $t \begin{pmatrix} 1 \ 0 \end{pmatrix}$ (for $t \ne 0$).

  • Conclusion: There is only one linearly independent eigenvector (e.g., $\begin{pmatrix} 1 \ 0 \end{pmatrix}$) for this matrix. We cannot form a basis of $R^2$ consisting of eigenvectors. Thus, $B$ is not diagonalizable.

• Key Criterion: Geometric vs. Algebraic Multiplicity

  • Algebraic Multiplicity (AM): The number of times an eigenvalue appears as a root of the characteristic polynomial.

  • Geometric Multiplicity (GM): The dimension of the eigenspace corresponding to an eigenvalue ($GM(\lambda) = \dim(E_{\lambda})$). This is the maximum number of linearly independent eigenvectors for that eigenvalue.

  • Bounds: For any eigenvalue $\lambda<em>i$, always: $1 \le GM(\lambda</em>i) \le AM(\lambda_i)$.

  • Theorem (General Diagonalizability Condition): A square matrix $A$ is diagonalizable if and only if for every eigenvalue $\lambda<em>i$ of $A$, its geometric multiplicity equals its algebraic multiplicity ($GM(\lambda</em>i) = AM(\lambda_i)$).

    • This means each eigenspace must be