Linear Systems and Matrices

Introduction to Linear Equations and Systems

• Definition of a Linear Equation:

  • A linear equation has all its variables of degree one only, meaning variables are only multiplied by constants.
  • The general standard form of a linear equation is:     $a_1 x_1 + a_2 x_2 + \dots + a_n x_n = b$
  • Components of the equation:
  • $x_1, x_2, \dots, x_n$ are the $n$ variables (unknowns).
  • $a_1, a_2, \dots, a_n$ and $b$ are constants.
  • Examples and classifications:
  • $2x + 3y + 4z = 3$: Linear equation with $3$ variables.
  • $3x^2 + y^2 = 1$: Non-linear (variables have degree $2$).
  • $xy = 1$: Non-linear (product of two variables).
  • $\sin(x) = 1$: Non-linear (trigonometric transcendental function of a variable).
  • $e^{x+y} = 2x$: Non-linear (exponential transcendental function).
  • $x + 5 = 7$: Linear equation with $1$ variable.
  • $x + 2y - 3z + 4w + 10h = 5$: Linear equation with $5$ variables.

• Definition of a Linear System (L.S):

  • A linear system ($\text{L.S}$) is a set of linear equations grouped together, each containing the same number of variables.
  • Solving a linear system means finding a specific value or set of values assigned to the variables that simultaneously satisfy all equations of the system when substituted.

• Worked Example 1: Solving via the Elimination Method (Unique Solution):

  • Consider the system:     $\begin{cases} 2x + 3y = 2 & (1) \\ x + 4y = 1 & (2) \end{cases}$
  • Multiply equation $(2)$ by $-2$ to produce equation $(3)$:     $-2x - 8y = -2 \quad (3)$
  • Add equations $(1)$ and $(3)$ together:     $(2x + 3y) + (-2x - 8y) = 2 + (-2) \implies -5y = 0 \implies y = 0$
  • Substitute $y = 0$ back into equation $(1)$:     $2x + 3(0) = 2 \implies 2x = 2 \implies x = 1$
  • The set of solutions is:     $S = \{(x, y) = (1, 0)\}$
  • This represents a unique solution.

• Worked Example 2: Inconsistent System (No Solution):

  • Consider the system:     $\begin{cases} 2x + 3y = 2 & (1) \\ 2x + 3y = 5 & (2) \end{cases}$
  • Subtract equation $(2)$ from equation $(1)$:     $(1) - (2) \implies 0 = -3$
  • This is impossible (a contradiction), meaning no solution exists.

• Classification of Linear Systems by Solution Count:

  • A linear system can have:
  • No solution (inconsistent system).
  • A unique solution (consistent system).
  • Infinitely many solutions (consistent system).
  • Definition of Consistency:
  • A consistent system is a linear system that admits at least one solution (either unique or infinitely many).
  • An inconsistent system is a linear system that admits no solution.
  • Example 1 is consistent; Example 2 is inconsistent.

• Worked Example 3: Consistent System with Infinitely Many Solutions:

  • Solve the system:     $\begin{cases} x_1 + 2x_2 - 3x_3 = -4 & (1) \\ 2x_1 + x_2 - 3x_3 = -4 & (2) \end{cases}$
  • Subtract equation $(2)$ from equation $(1)$:     $(1) - (2) \implies -x_1 + x_2 = 0 \implies x_2 = x_1$
  • Substitute $x_2 = x_1$ into equation $(1)$:     $x_1 + 2x_1 - 3x_3 = -4 \implies 3x_1 - 3x_3 = -4 \implies 3x_3 = 3x_1 + 4 \implies x_3 = x_1 + \frac{4}{3}$
  • Substitute into equation $(2)$ to verify consistency:     $2x_1 + x_1 - 3\left(x_1 + \frac{4}{3}\right) = -4 \implies 3x_1 - (3x_1 + 4) = -4 \implies 0x_1 - 4 = -4 \implies -4 = -4$
  • The identity holds for all values; thus $x_1$ is free to vary.
  • Setting $x_1 = t \in \mathbb{R}$, the full set of solutions is:     $S = \left\{(x_1, x_2, x_3) = \left(t, t, t + \frac{4}{3}\right), \, t \in \mathbb{R}\right\}$
  • The system is consistent with infinitely many solutions.
  • Sample particular solutions:
  • For $t = 0 \implies \left(0, 0, \frac{4}{3}\right)$
  • For $t = 1 \implies \left(1, 1, \frac{7}{3}\right)$

• Homogeneous Systems:

  • A homogeneous system is a linear system in which all equations have a second member (constant term on the right-hand side) equal to zero.
  • Trivial Solution:
  • The vector $(0, 0, \dots, 0)$ is always a solution to any homogeneous system, termed the trivial solution.
  • A homogeneous system can also admit non-trivial solutions (solutions where at least one variable is non-zero).
  • Every homogeneous system is consistent because it always admits at least the trivial solution.
  • Worked Example 4 (Homogeneous System with Infinitely Many Solutions):     $\begin{cases} x + 2y + z = 0 & (1) \\ x - 2y + z = 0 & (2) \\ 2x + 2z = 0 & (3) \end{cases}$
  • Add equations $(1)$ and $(2)$:       $(1) + (2) \implies 2x + 2z = 0 \quad (4)$
  • Subtract equation $(3)$ from equation $(4)$:       $(4) - (3) \implies 0x = 0 \text{ or } 0z = 0$
  • This indicates infinitely many solutions.
  • Let $x = t$. Then from equation $(3)$, $2z = -2x \implies z = -t$.
  • Substitute $x = t$ and $z = -t$ into equation $(1)$:       $t + 2y - t = 0 \implies 2y = 0 \implies y = 0$
  • Solution set:       $S = \{(x, y, z) = (t, 0, -t), \, t \in \mathbb{R}\}$
  • Particular instances:
    • $t = 0 \implies (0, 0, 0)$ (trivial solution).
    • $t = 1 \implies (1, 0, -1)$ (non-trivial solution).
    • $t = -2 \implies (-2, 0, 2)$ (non-trivial solution).
  • Worked Example 5 (Homogeneous System with a Unique Trivial Solution):     $\begin{cases} x + y + z = 0 & (1) \\ x + 2y - z = 0 & (2) \\ 2x + y + z = 0 & (3) \end{cases}$
  • Adding equations $(1)$ and $(2)$:       $2x + 3y = 0 \implies x = -\frac{3}{2}y$
  • Substitute $x = -\frac{3}{2}y$ into equation $(3)$:       $2\left(-\frac{3}{2}y\right) + y + z = 0 \implies -3y + y + z = 0 \implies z = 2y$
  • Substitute $x = -\frac{3}{2}y$ and $z = 2y$ into equation $(1)$:       $-\frac{3}{2}y + y + 2y = 0 \implies \frac{3}{2}y = 0 \implies y = 0$
  • Substituting $y = 0$ back yields $x = 0$ and $z = 0$.
  • The system admits only the unique trivial solution $(0, 0, 0)$.
  • Supplementary System for Practice:     $\begin{cases} 2x + 3y - z = 5 \\ -2x - 3y + z = -5 \\ 4x + 6y - 2z = 10 \end{cases}$

• Equivalent Linear Systems:

  • Two linear systems are said to be equivalent if and only if they possess the exact same set of solutions.

Matrices and Matrix Operations

• Definition and Notation of a Matrix:

  • A matrix $A$ is an array of data listed in rows and columns:     $A = \begin{pmatrix} a_{11} & a_{12} & \dots & a_{1n} \\ a_{21} & a_{22} & \dots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn} \end{pmatrix} = [a_{ij}]_{\substack{1 \le i \le m \\ 1 \le j \le n}}$
  • A matrix having $m$ rows and $n$ columns has dimension $m \times n$.
  • The entry $a_{ij}$ represents the coefficient or entry of $A$ located at the $i$-th row and $j$-th column.
  • Example Matrix:     $A = \begin{pmatrix} 1 & 2 & 3 & 4 \\ 2 & 1 & 0 & 2 \\ 0 & 0 & 1 & 2 \end{pmatrix}$
  • Dimension: $3 \times 4$.
  • Entries: $a_{22} = 1$, $a_{31} = 0$, $a_{14} = 4$.
  • The entry $a_{41}$ does not exist because there are only $3$ rows.
  • Space Notation: $M_{m,n}(\mathbb{R})$ denotes the set of all matrices of dimension $m \times n$ with entries belonging to $\mathbb{R}$.
  • Square Matrix: When $m = n$ (number of rows equals number of columns), the matrix is termed a square matrix of dimension $m$.
  • Main Diagonal: In any square matrix, the main diagonal is composed of the entries where the row index equals the column index:     $\{a_{11}, a_{22}, \dots, a_{mm}\}$
  • For example, in the $4 \times 4$ square matrix:       $A = \begin{pmatrix} 1 & 2 & 3 & 5 \\ 1 & 0 & 1 & 3 \\ 5 & 4 & 7 & 6 \\ 1 & 0 & 2 & 3 \end{pmatrix}$       The main diagonal consists of the entries: $1, 0, 7, 3$.

• Row and Column Vectors:

  • Let $A \in M_{m,n}(\mathbb{R})$:
  • If $m = 1$, $A$ is called a row vector. Example: $A = \begin{pmatrix} 1 & 2 & 0 & 5 \end{pmatrix} \in M_{1,4}(\mathbb{R})$.
  • If $n = 1$, $A$ is called a column vector. Example: $A = \begin{pmatrix} 2 \\ 2 \\ 5 \end{pmatrix} \in M_{3,1}(\mathbb{R})$.

• Overview of Fundamental Matrix Operations:

  • Equality: Defined between two matrices if and only if they share identical dimensions and all corresponding entries are equal.
  • Addition: Defined between two matrices if and only if they have the exact same dimension.
  • Scalar Multiplication: Can be performed for any scalar on any matrix regardless of dimension.
  • Transpose: Can be performed on any matrix of any dimension.
  • Multiplication: Constrained by dimension; the number of columns in the first matrix must match the number of rows in the second matrix.
  • Example Practice Problem:
  • Given $A = \begin{pmatrix} 1 & 2 & 5 \\ 0 & -1 & 1 \end{pmatrix}$ and $B = \begin{pmatrix} 2 & 0 & 2 \\ 1 & 1 & 1 \\ -1 & 0 & 7 \end{pmatrix}$.
  • Determine $(2A - 3B)^T$ and $(A + 2B)^T$ (noting dimension constraints).

• Definition and Examples of Matrix Trace:

  • Let $A$ be a square matrix of dimension $n$. The trace of $A$, denoted $\operatorname{Tr}(A)$, is the sum of all entries residing on the main diagonal:     $\operatorname{Tr}(A) = \sum_{i=1}^n a_{ii} = a_{11} + a_{22} + \dots + a_{nn}$
  • Examples:
  • For $A = \begin{pmatrix} 1 & 2 \\ -1 & 3 \end{pmatrix}$, $\operatorname{Tr}(A) = 1 + 3 = 4$.
  • For $A = \begin{pmatrix} 1 & 2 & 0 \\ 3 & 4 & 1 \\ -1 & 5 & -2 \end{pmatrix}$, $\operatorname{Tr}(A) = 1 + 4 + (-2) = 3$.

Algebraic Properties of Matrix Operations

• Properties of Matrix Addition:

  • Let $A, B, C \in M_{m,n}(\mathbb{R})$:
  • Commutativity: $A + B = B + A$
  • Associativity: $(A + B) + C = A + (B + C)$
  • Identity Element: There exists an $m \times n$ zero matrix $0_{m,n}$ such that:       $A + 0_{m,n} = 0_{m,n} + A = A$
  • Additive Inverse: There exists an $m \times n$ matrix $D = -A$ (called the negative of $A$) such that:       $A + D = D + A = 0_{m,n}$

• Properties of Scalar Multiplication:

  • Let $r, s \in \mathbb{R}$ and $A, B$ be matrices of appropriate dimensions:
  • Associativity of Scalars: $r \cdot (s \cdot A) = (r \cdot s) \cdot A$
  • Distributivity over Scalar Addition: $(r + s) \cdot A = r \cdot A + s \cdot A$
  • Distributivity over Matrix Addition: $r \cdot (A + B) = r \cdot A + r \cdot B$
  • Compatibility with Matrix Multiplication: $(r A) \cdot B = A \cdot (r B) = r \cdot (A B)$
  • Worked Example Demonstrating Scalar Associativity:
  • Let $A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$:       $2 \cdot (3 \cdot A) = 2 \cdot \begin{pmatrix} 3 & 6 \\ 9 & 12 \end{pmatrix} = \begin{pmatrix} 6 & 12 \\ 18 & 24 \end{pmatrix}$$(2 \cdot 3) \cdot A = 6 \cdot A = \begin{pmatrix} 6 & 12 \\ 18 & 24 \end{pmatrix}$
  • Both calculations produce identical results.

• Properties of Matrix Multiplication:

  • Associativity: $A \cdot (B \cdot C) = (A \cdot B) \cdot C$
  • Right Distributivity: $(A + B) \cdot C = A C + B C \ne C A + C B$ (the order of factors must be respected).

• Left Distributivity: $A \cdot (B + C) = A B + A C$

• Non-Commutativity: In general, matrix multiplication is not commutative:   $A B \ne B A \quad (\text{in general})$

• Properties of the Transpose Operation:

  • Let $r \in \mathbb{R}$ and $A, B$ be matrices of compatible sizes:
  • Involution: $(A^T)^T = A$
  • Transpose of a Sum: $(A + B)^T = A^T + B^T$
  • Transpose of Scalar Multiplication: $(r \cdot A)^T = r \cdot A^T$
  • Transpose of a Product: $(A \cdot B)^T = B^T \cdot A^T$

• Properties of the Trace:

  • Let $\lambda \in \mathbb{R}$ and $A, B \in M_n(\mathbb{R})$:
  • Scalar Linearity: $\operatorname{Tr}(\lambda A) = \lambda \operatorname{Tr}(A)$
  • Additive Linearity: $\operatorname{Tr}(A + B) = \operatorname{Tr}(A) + \operatorname{Tr}(B)$
  • Commutative Tracing of Products: $\operatorname{Tr}(A \cdot B) = \operatorname{Tr}(B \cdot A)$
  • Invariance under Transpose: $\operatorname{Tr}(A^T) = \operatorname{Tr}(A)$
  • Non-Negativity of Gram Trace: $\operatorname{Tr}(A \cdot A^T) \ge 0$

• Powers and Exponents of Square Matrices:

  • Let $A, B \in M_n(\mathbb{R})$ and $p, q \in \mathbb{N}$:
  • Definition: $A^p = \underbrace{A \times A \times \dots \times A}_{p\text{ times}}$
  • Index Law 1: $A^p \cdot A^q = A^{p+q}$
  • Index Law 2: $(A^p)^q = A^{p \cdot q}$
  • Power of a Commuting Product: If $A B = B A$, then $(A B)^p = A^p B^p$.
  • Derivation for $p = 3$ assuming $A B = B A$:       $(A B)^3 = (A B)(A B)(A B) = A(B A)(B A)B = A(A B)(A B)B = A \cdot A \cdot (A B) \cdot B \cdot B = A^3 B^3$

Special Matrix Types and Matrix Inversion

• The Identity Matrix:

  • The identity matrix of dimension $n$, denoted $I_n$, is the square matrix defined by:     $a_{ij} = \begin{cases} 0 & \text{if } i \ne j \\ 1 & \text{if } i = j \end{cases} \quad \forall \, 1 \le i, j \le n$
  • Explicit representations:     $I_2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}, \quad I_3 = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}$
  • Multiplicative Identity Property:
  • For square matrix $A \in M_n(\mathbb{R})$: $A \cdot I_n = I_n \cdot A = A$.
  • For rectangular matrix $A \in M_{m,n}(\mathbb{R})$: $A \cdot I_n = A$ and $I_m \cdot A = A$.

• The Inverse of a Matrix:

  • Definition: A square matrix $A$ of dimension $n$ is invertible (or non-singular) if and only if there exists a matrix $B \in M_n(\mathbb{R})$ such that:     $A \cdot B = B \cdot A = I_n$
  • The matrix $B$ is denoted by $A^{-1}$, known as the inverse of $A$.
  • Verification Example:
  • Show that $B = \begin{pmatrix} -1 & \frac{3}{2} \\ 1 & -1 \end{pmatrix}$ is the inverse of $A = \begin{pmatrix} 2 & 3 \\ 2 & 2 \end{pmatrix}$.
  • Compute the product $A \cdot B$:       $A \cdot B = \begin{pmatrix} 2 & 3 \\ 2 & 2 \end{pmatrix} \begin{pmatrix} -1 & \frac{3}{2} \\ 1 & -1 \end{pmatrix} = \begin{pmatrix} 2(-1) + 3(1) & 2\left(\frac{3}{2}\right) + 3(-1) \\ 2(-1) + 2(1) & 2\left(\frac{3}{2}\right) + 2(-1) \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} = I_2$
  • Thus, $B = A^{-1}$.
  • Fundamental consequence: $(A^{-1})^{-1} = B^{-1} = A$ (the inverse of the inverse is the original matrix).
  • Constructive Inversion Example using Systems of Equations:
  • Find the inverse of $A = \begin{pmatrix} 2 & 3 \\ 1 & 3 \end{pmatrix}$.
  • Let $A^{-1} = \begin{pmatrix} x & y \\ z & t \end{pmatrix}$. The requirement is $A \cdot A^{-1} = I_2$:       $\begin{pmatrix} 2 & 3 \\ 1 & 3 \end{pmatrix} \begin{pmatrix} x & y \\ z & t \end{pmatrix} = \begin{pmatrix} 2x + 3z & 2y + 3t \\ x + 3z & y + 3t \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$
  • This creates two independent systems of linear equations:       $\begin{cases} 2x + 3z = 1 & (1) \\ x + 3z = 0 & (2) \end{cases} \quad \text{and} \quad \begin{cases} 2y + 3t = 0 & (3) \\ y + 3t = 1 & (4) \end{cases}$
  • Solve the first pair:
    • $(1) - (2) \implies x = 1$
    • From $(2)$, $3z = -x = -1 \implies z = -\frac{1}{3}$
  • Solve the second pair:
    • $(3) - (4) \implies y = -1$
    • From $(3)$, $2(-1) + 3t = 0 \implies 3t = 2 \implies t = \frac{2}{3}$
  • Therefore, the inverse is:       $A^{-1} = \begin{pmatrix} 1 & -1 \\ -\frac{1}{3} & \frac{2}{3} \end{pmatrix}$

• Properties of Matrix Inverses:

  • Involution: $(A^{-1})^{-1} = A$
  • Uniqueness: If the inverse of a matrix exists, it is unique.
  • Product Inversion (Reversal of Order): If $A$ and $B$ are invertible matrices of dimension $n$, then $A B$ is invertible and:     $(A \cdot B)^{-1} = B^{-1} \cdot A^{-1}$
  • Transpose Inversion: If $A$ is invertible, then $A^T$ is invertible and:     $(A^T)^{-1} = (A^{-1})^T$
  • One-Sided Inverse Equivalence: Let $A \in M_n(\mathbb{R})$. If there exists a matrix $D \in M_n(\mathbb{R})$ such that $A \cdot D = I_n$, then there exists $C \in M_n(\mathbb{R})$ such that $C \cdot A = I_n$, and $C = D = A^{-1}$.

• Common Algebraic Pitfalls:

  • General Non-Commutativity: $A \cdot B \ne B \cdot A$

• Zero-Divisors Exist: A \cdot B = 0 \centernot\implies A = 0 \text{ or } B = 0

• Invalidation of General Cancellation: A B = A C \centernot\implies B = C

  • Cancellation is valid if and only if $A$ is invertible: If $A$ is invertible and $A B = A C$, then multiplying both sides by $A^{-1}$ yields $B = C$.
  • Counterexample demonstrating cancellation failure:     $A = \begin{pmatrix} 1 & 2 \\ 2 & 4 \end{pmatrix}, \quad B = \begin{pmatrix} 2 & 1 \\ 3 & 2 \end{pmatrix}, \quad C = \begin{pmatrix} -2 & 7 \\ 5 & -1 \end{pmatrix}$
  • Compute $A B$:       $A B = \begin{pmatrix} 1(2) + 2(3) & 1(1) + 2(2) \\ 2(2) + 4(3) & 2(1) + 4(2) \end{pmatrix} = \begin{pmatrix} 8 & 5 \\ 16 & 10 \end{pmatrix}$
  • Compute $A C$:       $A C = \begin{pmatrix} 1(-2) + 2(5) & 1(7) + 2(-1) \\ 2(-2) + 4(5) & 2(7) + 4(-1) \end{pmatrix} = \begin{pmatrix} 8 & 5 \\ 16 & 10 \end{pmatrix}$
  • Here $A B = A C$, but clearly $B \ne C$.

• Structural Classifications of Square Matrices:

  • Let $A = [a_{ij}] \in M_n(\mathbb{R})$:
  • Diagonal Matrix: $a_{ij} = 0$ whenever $i \ne j$.
    • Example: $\begin{pmatrix} 2 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 3 \end{pmatrix}$.
  • Upper Triangular Matrix: $a_{ij} = 0$ whenever $i > j$ (all entries strictly below the main diagonal are zero; entries on or above the diagonal are not necessarily zero).
  • Lower Triangular Matrix: $a_{ij} = 0$ whenever $i < j$ (all entries strictly above the main diagonal are zero).
  • Symmetric Matrix: A matrix equal to its transpose:       $A = A^T \iff a_{ij} = a_{ji} \quad \forall \, i, j$
    • The upper triangle mirrors the lower triangle; the main diagonal entries can be any values.
    • Example: $\begin{pmatrix} 1 & 3 & 5 \\ 3 & 2 & 0 \\ 5 & 0 & 1 \end{pmatrix}$.
  • Skew-Symmetric Matrix: A matrix equal to the negative of its transpose:       $A = -A^T \iff a_{ij} = -a_{ji} \quad \forall \, i, j$
    • For diagonal entries, $a_{ii} = -a_{ii} \implies 2a_{ii} = 0 \implies a_{ii} = 0$. Thus, all main diagonal entries of a skew-symmetric matrix must be zero.
    • Example: $\begin{pmatrix} 0 & 2 \\ -2 & 0 \end{pmatrix}$.

• Matrix Classification Examples:

  • $\begin{pmatrix} 1 & 0 & 3 \\ 0 & 0 & 2 \\ 0 & 0 & 1 \end{pmatrix}$: Upper triangular matrix.
  • $\begin{pmatrix} 1 & 2 & 3 \\ 2 & 0 & -1 \\ 3 & -1 & 0 \end{pmatrix}$: Symmetric matrix.
  • $\begin{pmatrix} 1 & 2 & 3 & 4 \\ 0 & 0 & 1 & 2 \\ 0 & 0 & 1 & 3 \end{pmatrix}$: Rectangular $3 \times 4$ matrix (neither triangular nor symmetric).
  • $\begin{pmatrix} 1 & 2 \\ -2 & 1 \end{pmatrix}$: Neither symmetric nor skew-symmetric (main diagonal is non-zero).
  • $\begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 \end{pmatrix}$: Diagonal matrix of dimension $4$.

• Theoretical Matrix Algebra Exercises:

  • Exercise 1: Let $A$ be a square matrix of dimension $n$. Prove that $A \cdot A^T$ is a symmetric matrix.
  • Proof: Using the transpose property of products, $(A \cdot A^T)^T = (A^T)^T \cdot A^T = A \cdot A^T$. Because the matrix equals its own transpose, it is symmetric.
  • Exercise 2: Prove or disprove each statement:
  • a) If $A \cdot B = B \cdot A$, then $A^2 B = B A^2$.
  • b) For any $n \times n$ matrix $A$, $A^T \cdot A = A \cdot A^T$.
  • c) If $A$ is invertible, then there exists $B \ne 0$ such that $A \cdot B = 0$.

Row Echelon Forms and Elementary Operations

• Definition of Row Echelon Form (R.E.F):

  • An $m \times n$ matrix is in Row Echelon Form (R.E.F) if it meets the following criteria:
  • a) All rows consisting entirely of zeros (zero rows), if any exist, are grouped at the very bottom of the matrix.
  • b) The first non-zero entry of any non-zero row (reading from left to right) is equal to $1$. This entry is termed the leading one (or pivot) of the row.
  • c) For any two successive non-zero rows $i$ and $i+1$, the leading one of row $i+1$ appears strictly to the right of the leading one of row $i$.
  • As a necessary consequence, in any matrix in R.E.F, all entries lying directly below each leading one are zero.

• Definition of Reduced Row Echelon Form (R.R.E.F):

  • An $m \times n$ matrix is in Reduced Row Echelon Form (R.R.E.F) if it satisfies conditions (a), (b), and (c) of R.E.F, plus the following additional condition:
  • d) All entries appearing both directly below and directly above each leading one are zero (each leading one is the only non-zero entry in its entire column).

• Examples of Form Classifications:

  • Matrix a:     $\begin{pmatrix} 1 & 2 & 0 & 4 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 1 & -3 \end{pmatrix}$
  • Not in R.E.F because a zero row is situated above a non-zero row.
  • Matrix b:     $\begin{pmatrix} 1 & 2 & 3 & 4 \\ 0 & 1 & -2 & 5 \\ 0 & 0 & 1 & 2 \\ 0 & 0 & 0 & 0 \end{pmatrix}$
  • Satisfies all conditions for R.E.F.
  • Matrix c:     $\begin{pmatrix} 0 & 0 & 1 & 3 & 5 & 7 & 9 \\ 0 & 0 & 0 & 0 & 1 & -2 & 3 \\ 0 & 0 & 0 & 0 & 0 & 1 & 2 \\ 0 & 0 & 0 & 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{pmatrix}$
  • Satisfies all conditions for R.E.F.

• The Three Elementary Row Operations:

  • 1) Interchange two rows: $R_i \leftrightarrow R_j$.
  • 2) Multiply a row by a non-zero scalar: $R_i \leftarrow k R_i$ where $k \ne 0$.
  • 3) Add a scalar multiple of one row to another row: $R_i \leftarrow R_i + k R_j$.

• Systematic Reduction Algorithm to R.E.F and R.R.E.F:

  • Step 1: Search for the pivot column, which is the first column from the left that does not consist entirely of zeros.
  • Step 2: Identify the first non-zero entry in the pivot column. If this entry is not at the top of the current working block, swap rows to place it at the top. This entry serves as the pivot. Scale the row by $\frac{1}{\text{pivot}}$ to turn it into a leading one.
  • Step 3: Apply row addition operations using the leading one row to clear all entries directly beneath the pivot to zero. Repeat Steps 1 through 3 on the remaining submatrix below and to the right until the matrix achieves R.E.F.
  • Step 4 (To reach R.R.E.F): Starting with the rightmost leading one and moving backwards, use elementary row additions to eliminate all non-zero entries lying above the leading ones.

• Row Equivalence and Matrix Rank:

  • Row Equivalence: An $m \times n$ matrix $B$ is row equivalent to an $m \times n$ matrix $A$ if $B$ can be obtained from $A$ by applying a finite sequence of elementary row operations.
  • Matrix Rank: Given an $m \times n$ matrix $A$, the rank of $A$, denoted $\operatorname{rank}(A)$, is the total number of leading ones present in the matrix after it has been fully transformed into R.E.F.
  • Practice Transformation Matrix:     $A = \begin{pmatrix} 0 & 2 & 1 & 1 & 3 & 7 \\ 5 & 3 & 1 & 2 & 0 & 0 \\ 0 & 0 & 0 & 1 & 1 & 3 \\ 0 & 0 & 0 & 0 & 0 & 0 \\ 1 & 2 & 1 & 1 & 0 & 1 \end{pmatrix}$
  • Transform this matrix into R.E.F and R.R.E.F to determine $\operatorname{rank}(A)$.

Matrix Representation of Systems and Solving Methods

• Formulating Linear Systems as Matrix Equations:

  • Consider a general system $(S)$ of $m$ equations and $n$ unknowns with coefficients in $\mathbb{R}$ (or $\mathbb{C}$), where $m, n \ge 1$:     $(S): \begin{cases} a_{11} x_1 + a_{12} x_2 + \dots + a_{1n} x_n = b_1 \\ a_{21} x_1 + a_{22} x_2 + \dots + a_{2n} x_n = b_2 \\ \vdots \\ a_{m1} x_1 + a_{m2} x_2 + \dots + a_{mn} x_n = b_m \end{cases}$
  • Component Definitions:
  • Coefficient Matrix ($m \times n$):       $A = \begin{pmatrix} a_{11} & a_{12} & \dots & a_{1n} \\ a_{21} & a_{22} & \dots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn} \end{pmatrix}$
  • Unknown Vector ($n \times 1$):       $X = \begin{pmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{pmatrix}$
  • Second Member / Right-Hand Side Vector ($m \times 1$):       $B = \begin{pmatrix} b_1 \\ b_2 \\ \vdots \\ b_m \end{pmatrix}$
  • Compact Matrix Equation:     $A \cdot X = B$
  • The product $A \cdot X$ is well-defined and has dimension $m \times 1$.
  • Example System:     $\begin{cases} 2x_1 + 3x_2 - 4x_3 + x_4 = 5 \\ -2x_1 + x_3 = 7 \\ 3x_1 + 2x_2 - 4x_4 = 3 \end{cases}$
  • Coefficient matrix $A$ of dimension $3 \times 4$:       $A = \begin{pmatrix} 2 & 3 & -4 & 1 \\ -2 & 0 & 1 & 0 \\ 3 & 2 & 0 & -4 \end{pmatrix}$
  • Unknown vector $X$ of dimension $4 \times 1$:       $X = \begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \end{pmatrix}$
  • Constant vector $B$ of dimension $3 \times 1$:       $B = \begin{pmatrix} 5 \\ 7 \\ 3 \end{pmatrix}$

• The Augmented Matrix:

  • The augmented matrix associated with system $(S)$ is denoted $(A \mid B)$, combining coefficients and constants:     $(A \mid B) = \begin{pmatrix} a_{11} & a_{12} & \dots & a_{1n} & b_1 \\ a_{21} & a_{22} & \dots & a_{2n} & b_2 \\ \vdots & \vdots & \ddots & \vdots & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn} & b_m \end{pmatrix}$
  • Critical Rule: Only elementary row operations are permitted on augmented matrices; column operations alter the system relationships and are strictly forbidden.

• Matrix-Based Methods for Solving Linear Systems:

  • 1) Gaussian Elimination: Transform the augmented matrix $(A \mid B)$ into R.E.F, followed by back-substitution to solve for each variable.
  • 2) Gauss-Jordan Reduction: Transform the augmented matrix $(A \mid B)$ directly into R.R.E.F, from which solution values can be read immediately without back-substitution.
  • 3) Matrix Inversion ($A^{-1}$): Applicable exclusively when $A$ is square and invertible:     $A X = B \implies A^{-1}(A X) = A^{-1} B \implies (A^{-1} A) X = A^{-1} B \implies X = A^{-1} B$
  • This directly computes the unique solution via a single matrix product.
  • 4) Cramer's Rule: Applicable when $A$ is square and invertible, utilizing determinants of coefficient replacement matrices (studied after determinants).

• The Rouché-Capelli Theorem:

  • A linear system $A X = B$ is consistent if and only if the rank of the coefficient matrix equals the rank of the augmented matrix:     $\text{A linear system } A X = B \text{ is consistent} \iff \operatorname{rank}(A) = \operatorname{rank}(A \mid B)$

Invertibility Theorems and Algorithms

• Equivalence Theorem for Invertible Matrices:

  • Let $A$ be a square matrix of dimension $n \times n$. The following statements are mathematically equivalent:
  • $A$ is non-singular (invertible).
  • $A$ is row equivalent to the identity matrix $I_n$ ($A \sim I_n$).
  • $\operatorname{rank}(A) = n$.
  • The linear system $A X = B$ admits a unique solution for every vector $B$, given by $X = A^{-1} B$.

• Methods to Compute $A^{-1}$:

  • Method 1: Gauss-Jordan Elimination on Augmented Block Matrix:
  • Form the block matrix of size $n \times 2n$:       $(A \mid I_n)$
  • Apply elementary row operations to reduce the left block into R.R.E.F:
    • If $A$ reduces to $I_n$:         $(A \mid I_n) \xrightarrow{\text{R.R.E.F}} (I_n \mid B) \implies B = A^{-1}$
    • If the left side reduces to a matrix $C \ne I_n$ (possessing a zero row):         $(A \mid I_n) \xrightarrow{\text{R.R.E.F}} (C \mid B) \implies A \text{ is not invertible}$

• Method 2: Determinants and Adjoint Matrices:

  • Involves the formula $A^{-1} = \frac{1}{\det(A)} \operatorname{adj}(A)$, evaluated after Chapter 2.

Problem Sets and Exercises

• Exercise Set I: Solvability and Parameterized Linear Systems:

  • 1) Solve the following systems of equations:
  • System a:       $\begin{cases} x_1 + x_2 + 2x_3 = -1 \\ x_1 - 2x_2 + x_3 = -5 \\ 3x_1 + x_2 + x_3 = 3 \end{cases}$
  • System b:       $\begin{cases} x + y + z + w = 0 \\ x + w = 0 \\ x + 2y + z = 0 \end{cases}$
  • 2) Determine all values of the real parameter $a$ for which each system has (i) no solution, (ii) a unique solution, or (iii) infinitely many solutions:
  • System a (Ex 1.24):       $\begin{cases} x + y + z = 2 \\ 2x + 3y + 2z = 5 \\ 2x + 3y + (a^2 - 1)z = a + 1 \end{cases}$
  • System b:       $\begin{cases} x + 2y - 3z = 4 \\ 3x - y + 5z = 2 \\ 4x + y + (a^2 - 14)z = a + 2 \end{cases}$
  • 3) Determine the values of $a$ and the relation between $b$ and $c$ such that the system admits (i) no solution, (ii) a unique solution, or (iii) infinitely many solutions:     $\begin{cases} x + y - z = a \\ x - y + 2z = b \\ 3x + y + a z = c \end{cases}$
  • Assigned supplementary problems: Exercises 1.22, 1.24, 1.25.

• Exercise Set II: Matrix Inversion and System Applications:

  • 1) Inversion and Deduction:
  • a) Find the inverse of the matrix:       $A = \begin{pmatrix} 1 & 1 & 1 \\ 1 & 2 & 1 \\ 1 & 1 & 2 \end{pmatrix}$
  • b) Deduce the solution to the linear system:       $\begin{cases} x + y + z = 1 \\ x + 2y + z = 2 \\ x + y + 2z = 3 \end{cases}$
  • 2) Comparative System Solving:
  • Let $A = \begin{pmatrix} 2 & 1 & 1 \\ 1 & 1 & 0 \\ 2 & 1 & 3 \end{pmatrix}$ and $B = \begin{pmatrix} 6 \\ 3 \\ 8 \end{pmatrix}$.
  • a) Find $A^{-1}$ if it exists.
  • b) Determine whether the system $A X = B$ admits infinitely many solutions, a unique solution, or no solution.
  • c) Solve the system $A X = B$ using two different methods.
  • 3) Parameterized Matrix Invertibility:
  • Find all values of $a$ for which the inverse of $A$ exists, and determine $A^{-1}$ when it exists:       $A = \begin{pmatrix} 1 & 1 & 0 \\ 1 & 0 & 0 \\ 1 & 2 & a \end{pmatrix}$
  • 4) Matrix Equation Solving:
  • Let $A, B, C$ be invertible matrices. Solve for the matrix $D$ given that:       $A \cdot B \cdot C^T \cdot D \cdot B \cdot A^T \cdot C = A \cdot B^T$