06/24 Lecture Inverses of Matrices and LU Factorization Lecture Notes

Introduction to Matrix Inverses (Section 3.3)

• Focus of Study: The primary interest in this section is the inverse of square matrices, specifically those of order $n \times n$.

• Definitions and Terminology:

  • Invertible: This is a "fancy name" for being able to find the inverse of a matrix or being able to "invert" a matrix.

  • Invertibility Condition: Not every square matrix is invertible. A square matrix must satisfy specific conditions to have an inverse.

  • Notation: The inverse of a matrix $A$ is denoted as $A^{-1}$. The transcript notes that some older or specific texts might refer to it as $A'$, but $A^{-1}$ is the industry standard.

  • Clarification of Notation: The notation $A^{-1}$ does not mean $\frac{1}{A}$. While in common algebra $x^{-1} = \frac{1}{x}$, this rule does not apply to matrices, as matrix division is not defined in that manner.

Fundamental Principles of Inverses

• The Identity Relationship: In standard algebra, $x \times x^{-1} = 1$. In linear algebra, if matrix $A$ has an inverse $A^{-1}$, then the product of the two is the identity matrix ($I$):

  • $A \times A^{-1} = I$

  • $A^{-1} \times A = I$

• The Identity Matrix ($I$): This is the matrix equivalent of the number 1 in scalar algebra. It consists of ones along the main diagonal and zeros everywhere else.

• Commutative Property of Inverses: Generally, matrix multiplication is not commutative ($AB \neq BA$). However, the product of a matrix and its inverse is one of the rare occasions where the order does not matter: both $A \cdot A^{-1}$ and $A^{-1} \cdot A$ result in the identity matrix of the same order ($I_n$).

Non-Invertible Matrices and Determinants

• Zero Matrices: A matrix consisting entirely of zeros does not have an inverse. For any matrix $B$, the product of the zero matrix and $B$ will result in a zero matrix, which can never equal the identity matrix ($I$).

• Dependent Rows/Columns: If a matrix has rows or columns that are multiples of one another (linearly dependent), the matrix will not have an inverse.

  • Example from Transcript: Consider the matrix:     $\begin{pmatrix} 1 & 2 \\ 2 & 4 \end{pmatrix}$     This matrix corresponds to dependent equations (the second row is twice the first). Its determinant is zero, thus making it non-invertible.

• Determinant Condition: For a square matrix to be invertible, its determinant ($Det(A)$) must not be zero:

  • $Det(A) \neq 0 \rightarrow \text{Invertible}$

  • $Det(A) = 0 \rightarrow \text{Not Invertible (Singular)}$

Uniqueness of the Inverse

• Theorem of Uniqueness: If a square matrix has an inverse, that inverse is unique. A single matrix cannot have two different inverses.

• Proof Logic:

  • Suppose matrix $A$ has two inverses, $A_1$ and $A_2$.

  • Then $A \times A_1 = I$ and $A \times A_2 = I$.

  • Therefore, $A \times A_1 = A \times A_2$.

  • Multiplying both sides on the left by the inverse of $A$ (which is assumed to exist), the $A$ terms cancel out to becomes $I$, leaving $A_1 = A_2$.

Solving Linear Systems Using Inverses

• Algebraic Analogy: If $ax = b$, then $x = a^{-1}b$.

• Application to Matrices: For a system defined by the matrix equation $Ax = b$ (where $x$ and $b$ are vectors):

  1. Start with $Ax = b$.

  2. Left-multiply both sides by $A^{-1}$: $A^{-1}(Ax) = A^{-1}b$.

  3. Simplify using the associative property: $(A^{-1}A)x = A^{-1}b$.

  4. Since $A^{-1}A = I$, then $Ix = A^{-1}b$.

  5. The solution is $x = A^{-1}b$.

• Limitations: This method only provides a unique solution if $A$ is invertible. If $A$ is not invertible, the system may have zero solutions (parallel lines) or infinitely many solutions (coincidental lines/free variables).

Inverse of a $2 \times 2$ Matrix

• Standard Form: For a matrix $A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}$.

• Determinant: First calculate the determinant: $\Delta = ad - bc$.

• Formula for Inverse: If $ad - bc \neq 0$, then:     $A^{-1} = \frac{1}{ad - bc} \begin{pmatrix} d & -b \\ -c & a \end{pmatrix}$

• Procedure:

  1. Swap the elements on the main diagonal ($a$ and $d$).

  2. Change the signs of the elements on the minor diagonal ($b$ and $c$).

  3. Multiply the resulting matrix by the scalar $\frac{1}{\text{Determinant}}$.

Numerical Examples of Inverse Calculation

• Example 1: Invertible Matrix

  • Matrix $A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$

  • Determinant: $(1 \times 4) - (3 \times 2) = 4 - 6 = -2$.

  • Preliminary Inverse Shape: $\frac{1}{-2} \begin{pmatrix} 4 & -2 \\ -3 & 1 \end{pmatrix}$.

  • Final Inverse Matrix: $A^{-1} = \begin{pmatrix} -2 & 1 \\ 1.5 & -0.5 \end{pmatrix}$.

• Example 2: Non-Invertible Matrix

  • Matrix $B = \begin{pmatrix} -16 & -60 \\ ? & ? \end{pmatrix}$ (Referenced in context of row dependency).

  • If rows are multiples of each other, the determinant is zero. For instance, if row one multiplied by 3 gives row two, the matrix is singular.

Algebraic Properties of Inverses

• Property 1: Double Inversion

  • $(A^{-1})^{-1} = A$

  • Inverting an inverse matrix returns the original matrix (like "double negation" in logic).

• Property 2: Scalar Multiplication

  • If $c$ is a non-zero scalar, then $(cA)^{-1} = \frac{1}{c} A^{-1}$.

• Property 3: Inverse of a Product (The Socks-and-Shoes Rule)

  • $(AB)^{-1} = B^{-1} A^{-1}$

  • The inverse of a product is the product of the inverses in reverse order.

  • Proof: To solve $ABx = I$ for $x$, left-multiply by $A^{-1}$ to get $Bx = A^{-1}$, then left-multiply by $B^{-1}$ to get $x = B^{-1} A^{-1}$.

• Property 4: Transposition

  • $(A^T)^{-1} = (A^{-1})^T$

  • The inverse of the transpose is the transpose of the inverse.

• Property 5: Powers

  • $(A^n)^{-1} = (A^{-1})^n = A^{-n}$, for non-negative integers $n$.

Solving Matrix Equations for $x$

• Scenario: Solve for $x$ given an equation with multiple matrix variables.

• Example Process:

  • If given $A^{-1}(BX)^{-1} = A^{-3}A^{-1}B^6$.

  • Applying the product inverse rule: $(BX)^{-1} = X^{-1}B^{-1}$.

  • Multiplying both sides by the inverse of the lead matrix (e.g., multiplying by $A$ to cancel $A^{-1}$).

  • The transcript describes complex manipulations resulting in solutions such as $x = B^{-7} A^4$ or variations depending on the initial specificities of the given identity operations.

Elementary Matrices

• Definition: An elementary matrix ($E$) is a matrix obtained by performing a single elementary row operation on an identity matrix ($I$).

• Three Elementary Row Operations (EROs):

  1. Row Swap: Swapping two rows ($R_i \leftrightarrow R_j$).

  2. Row Scaling: Multiplying a row by a non-zero constant ($k R_i$).

  3. Row Addition: Adding a multiple of one row to another ($k R_i + R_j \rightarrow R_j$).

• Properties of Elementary Matrices:

  • Every elementary matrix is invertible.

  • The inverse of an elementary matrix is also an elementary matrix that "undoes" the original operation.

  • Scaling a row by $k$ has an inverse of scaling by $\frac{1}{k}$.

  • The inverse of a row swap is the same row swap.

  • The inverse of adding $k R_i$ to $R_j$ is subtracting $k R_i$ from $R_j$ (adding $-k R_i$).

Gauss-Jordan Elimination Method for Finding Inverses

• The Augmented Matrix Method: This is used for $3 \times 3$ matrices or larger.

  1. Create an augmented matrix of the form $[A | I]$, where $I$ is the identity matrix of the same order.

  2. Perform elementary row operations to reduce the left side ($A$) to the identity matrix ($I$).

  3. Apply the exact same operations to the right side ($I$).

  4. When the left side becomes $I$, the right side will have transformed into $A^{-1}$.

  5. Resulting Form: $[I | A^{-1}]$.

• Checking for Invertibility: If during the process a row of all zeros appears on the left side, the matrix is not invertible (not have an inverse).

LU Factorization (Section 3.4)

• Definition: LU factorization (or decomposition) breaks a square matrix $A$ into the product of two triangular matrices:

  • L (Lower Triangular): A matrix with ones on the main diagonal and zeros above the diagonal.

  • U (Upper Triangular): A matrix in row-echelon form, with zeros below the main diagonal.

• Mathematical Concept: If row operations represented by elementary matrices $E_1, E_2, E_3$ convert matrix $A$ into an upper triangular matrix $U$, then:

  • $E_3 E_2 E_1 A = U$

  • $A = (E_1^{-1} E_2^{-1} E_3^{-1}) U$

  • The product $(E_1^{-1} E_2^{-1} E_3^{-1})$ corresponds to the lower triangular matrix $L$.

• Purpose: This method is useful in computer science and numerical analysis for efficiently solving systems of equations.

Class Discussion and Logistics

• Technology Note: The instructor mentions that modern technology (calculators/computers) can handle the inverse of massive matrices, such as those of order $1000 \times 1000$, though student work is restricted mainly to $2 \times 2$ or $3 \times 3$ for manual calculations.

• Homework Problems Mentioned:

  • Number 32, 33 on page 160.

  • Number 35 (a matrix appearing to be an elementary matrix derived from adding row 3 to row 2).

  • Number 53 (a $3 \times 3$ matrix that, upon Gaussian elimination, resulted in a row of zeros, concluding it is not invertible).

• Break Time: The class took a 10-minute break starting at 11:15 AM and reconvened at 11:25 AM.

• Attendance: At the end of the lecture, the instructor attempted to navigate the official class portal to record attendance.