Comprehensive Study Notes on Matrix Equations, Row Reduction, Solution Existence, and Homogeneous Systems

Algebraic Properties of Matrix-Vector Multiplication

For an $m \times n$ matrix $A$ and vectors $\mathbf{u}, \mathbf{v} \in \mathbb{R}^n$, along with a scalar $c$, matrix-vector multiplication satisfies the following two fundamental properties:

• Distributive Property of Matrix Addition over Vector Addition:   $A(\mathbf{u} + \mathbf{v}) = A\mathbf{u} + A\mathbf{v}$
• Commutativity of Scalar Multiplication:   $A(c\mathbf{u}) = c(A\mathbf{u})$

Proof of $A(\mathbf{u} + \mathbf{v}) = A\mathbf{u} + A\mathbf{v}$

• Let $A$ be a matrix represented in terms of its column vectors:   $A = \begin{pmatrix} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_3 \end{pmatrix}$
• Let $\mathbf{u}$ and $\mathbf{v}$ be vectors in $\mathbb{R}^3$ with scalar components:   $\mathbf{u} = \begin{pmatrix} u_1 \\ u_2 \\ u_3 \end{pmatrix}, \quad \mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \\ v_3 \end{pmatrix}$
• By Definition 1 (the definition of vector addition), the sum $\mathbf{u} + \mathbf{v}$ is computed component-wise:   $\mathbf{u} + \mathbf{v} = \begin{pmatrix} u_1 + v_1 \\ u_2 + v_2 \\ u_3 + v_3 \end{pmatrix}$
• Applying the definition of matrix-vector multiplication as a linear combination of the columns of $A$:   $A(\mathbf{u} + \mathbf{v}) = (u_1 + v_1)\mathbf{a}_1 + (u_2 + v_2)\mathbf{a}_2 + (u_3 + v_3)\mathbf{a}_3$
• By the algebraic properties of vectors, distribute each scalar sum across the respective column vectors:   $A(\mathbf{u} + \mathbf{v}) = u_1\mathbf{a}_1 + v_1\mathbf{a}_1 + u_2\mathbf{a}_2 + v_2\mathbf{a}_2 + u_3\mathbf{a}_3 + v_3\mathbf{a}_3$
• Regroup and reorder the terms using the commutative and associative properties of vector addition to isolate the $\mathbf{u}$ components from the $\mathbf{v}$ components:   $A(\mathbf{u} + \mathbf{v}) = (u_1\mathbf{a}_1 + u_2\mathbf{a}_2 + u_3\mathbf{a}_3) + (v_1\mathbf{a}_1 + v_2\mathbf{a}_2 + v_3\mathbf{a}_3$
• Re-applying the definition of matrix-vector multiplication to each grouped expression yields:   $A(\mathbf{u} + \mathbf{v}) = A\mathbf{u} + A\mathbf{v}$

Proof of $A(c\mathbf{u}) = c(A\mathbf{u})$

• By the definition of scalar multiplication of a vector, scaling $\mathbf{u}$ by $c$ gives:   $c\mathbf{u} = \begin{pmatrix} c u_1 \\ c u_2 \\ c u_3 \end{pmatrix}$
• Applying the definition of matrix-vector multiplication, express $A(c\mathbf{u})$ as a linear combination of the column vectors of $A$:   $A(c\mathbf{u}) = (c u_1)\mathbf{a}_1 + (c u_2)\mathbf{a}_2 + (c u_3)\mathbf{a}_3$
• Factor out the common scalar $c$ from the linear combination using vector space properties:   $A(c\mathbf{u}) = c (u_1\mathbf{a}_1 + u_2\mathbf{a}_2 + u_3\mathbf{a}_3)$
• Substitute the linear combination inside the parentheses with its matrix-vector multiplication definition $A\mathbf{u}$:   $A(c\mathbf{u}) = c(A\mathbf{u})$

Equivalence of Matrix Equations, Vector Equations, and Linear Systems

• Fundamental Theorem on Matrix Equations:

  • If $A$ is an $m \times n$ matrix with columns $\mathbf{a}_1, \mathbf{a}_2, \dots, \mathbf{a}_n$, and if $\mathbf{b} \in \mathbb{R}^m$, then the matrix equation:     $A\mathbf{x} = \mathbf{b}$
  • Has the exact same solution set as the vector equation:     $x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \dots + x_n\mathbf{a}_n = \mathbf{b}$
  • Which in turn has the exact same solution set as the system of linear equations whose augmented matrix is given by:     $\begin{pmatrix} \mathbf{a}_1 & \mathbf{a}_2 & \dots & \mathbf{a}_n & \mathbf{b} \end{pmatrix}$

• Equivalent Formulations of System Consistency:

  • The following statements are fully equivalent for a matrix equation $A\mathbf{x} = \mathbf{b}$:
  1. The equation $A\mathbf{x} = \mathbf{b}$ has a solution (i.e., is consistent).
  2. The target vector $\mathbf{b}$ is expressible as a linear combination of the column vectors of $A$
  3. The vector $\mathbf{b}$ belongs to the set spanned by the columns of $A$, denoted as $\mathbf{b} \in \text{Span}\{\mathbf{a}_1, \mathbf{a}_2, \dots, \mathbf{a}_n\}$.

Row Reduction and System Consistency Analysis

• Problem Setup:

  • Consider a $3 \times 3$ matrix $A$ and a right-hand side vector $\mathbf{b} \in \mathbb{R}^3$:     $A = \begin{pmatrix} 1 & 3 & 4 \\ -4 & 2 & -6 \\ -3 & -2 & -7 \end{pmatrix}, \quad \mathbf{b} = \begin{pmatrix} b_1 \\ b_2 \\ b_3 \end{pmatrix}$
  • Construct the corresponding augmented matrix $\begin{pmatrix} A & \mathbf{b} \end{pmatrix}$:     $\begin{pmatrix} 1 & 3 & 4 & b_1 \\ -4 & 2 & -6 & b_2 \\ -3 & -2 & -7 & b_3 \end{pmatrix}$

• Step-by-Step Row Reduction Algorithm:

  • Identify the leftmost non-zero column (Column 1) and select the top-left entry $1$ as the first pivot position.
  • Eliminate all entries below the first pivot:
  • Replace Row 2 with $R_2 + 4 R_1 \rightarrow R_2$:
    • First entry: $-4 + 4(1) = 0$
    • Second entry: $2 + 4(3) = 14$
    • Third entry: $-6 + 4(4) = 10$
    • Fourth entry: $b_2 + 4 b_1 = 4b_1 + b_2$
  • Replace Row 3 with $R_3 + 3 R_1 \rightarrow R_3$:
    • First entry: $-3 + 3(1) = 0$
    • Second entry: $-2 + 3(3) = 7$
    • Third entry: $-7 + 3(4) = 5$
    • Fourth entry: $b_3 + 3 b_1 = 3b_1 + b_3$
  • Intermediate augmented matrix:     $\begin{pmatrix} 1 & 3 & 4 & b_1 \\ 0 & 14 & 10 & 4b_1 + b_2 \\ 0 & 7 & 5 & 3b_1 + b_3 \end{pmatrix}$
  • Eliminate the entry in Row 3 under the second column pivot using the operation $R_3 + \left(-\frac{1}{2}\right) R_2 \rightarrow R_3$:
  • First entry: $0 + \left(-\frac{1}{2}\right)(0) = 0$
  • Second entry: $7 + \left(-\frac{1}{2}\right)(14) = 0$
  • Third entry: $5 + \left(-\frac{1}{2}\right)(10) = 0$
  • Fourth entry: $(3b_1 + b_3) - \frac{1}{2}(4b_1 + b_2) = 3b_1 + b_3 - 2b_1 - \frac{1}{2}b_2 = b_1 - \frac{1}{2}b_2 + b_3$
  • Final Row Echelon Form Matrix:     $\begin{pmatrix} 1 & 3 & 4 & b_1 \\ 0 & 14 & 10 & 4b_1 + b_2 \\ 0 & 0 & 0 & b_1 - \frac{1}{2}b_2 + b_3 \end{pmatrix}$

• Analysis of Solution Set and Consistency Conditions:

  • A system in echelon form is consistent if and only if there is no row of the form $\begin{pmatrix} 0 & 0 & 0 & d \end{pmatrix}$ where $d \neq 0$.
  • In this reduced matrix, the bottom row imposes the necessary and sufficient consistency condition:     $b_1 - \frac{1}{2}b_2 + b_3 = 0$
  • If the components $b_1, b_2, b_3$ do not satisfy this exact algebraic relation, the system has no solution.
  • Conclusion: The system $A\mathbf{x} = \mathbf{b}$ is not consistent for all arbitrary vectors $\mathbf{b} \in \mathbb{R}^3$.

Theorem on Existence of Solutions

• Statement of the Equivalent Conditions Theorem:

  • Let $A$ be an $m \times n$ matrix. The following four statements are logically equivalent (they are either all true or all false for any given matrix $A$):
  1. For each vector $\mathbf{b} \in \mathbb{R}^m$, the equation $A\mathbf{x} = \mathbf{b}$ has a solution.
  2. Each vector $\mathbf{b} \in \mathbb{R}^m$ is a linear combination of the columns of $A$
  3. The columns of $A$ span $\mathbb{R}^m$
  4. The matrix $A$ has a pivot position in every single row.

• Structural Proof Logic & Operational Verification:

  • Let $U$ denote the row echelon form of $A$
  • If Statement 4 holds true ($A$ has a pivot position in each row), then $U$ has no zero rows.
  • When row-reducing the augmented matrix $\begin{pmatrix} A & \mathbf{b} \end{pmatrix}$ to $\begin{pmatrix} U & \mathbf{f} \end{pmatrix}$, no row can feature a pivot position in the rightmost column $\mathbf{f}$, because every row already contains a pivot position within $U$. A single row cannot hold two pivot positions.
  • Since $\begin{pmatrix} U & \mathbf{f} \end{pmatrix}$ contains no inconsistent rows of the form $\begin{pmatrix} 0 & 0 & \dots & 0 & d \end{pmatrix}$ with $d \neq 0$, the system $A\mathbf{x} = \mathbf{b}$ is guaranteed to have a solution for every $\mathbf{b}$.
  • Conversely, if Statement 4 fails ($A$ does not have a pivot in every row), $U$ contains at least one row of zeros. Choosing a vector $\mathbf{f}$ with a non-zero entry in that corresponding row (such as $\mathbf{f} = \begin{pmatrix} 0 \\ \vdots \\ 0 \\ 1 \end{pmatrix}$) produces an inconsistent system $0 = 1$.
  • Practical Importance: Statement 4 provides an efficient computational test. Checking every arbitrary vector $\mathbf{b}$ directly is impossible due to the infinite size of $\mathbb{R}^m$, but row-reducing $A$ to verify pivot positions in every row easily determines consistency for all $\mathbf{b}$.

Homogeneous Linear Systems

• Definition:

  • A linear system is classified as homogeneous if it can be expressed in the form:     $A\mathbf{x} = \mathbf{0}$     where $A$ is an $m \times n$ matrix and $\mathbf{0}$ is the zero vector in $\mathbb{R}^m$

• Trivial and Non-Trivial Solutions:

  • The zero vector $\mathbf{x} = \mathbf{0}$ is always a solution to any homogeneous system $A\mathbf{x} = \mathbf{0}$, called the trivial solution.
  • A non-trivial solution is defined as any non-zero vector $\mathbf{x} \neq \mathbf{0}$ that satisfies $A\mathbf{x} = \mathbf{0}$.
  • Consistent linear systems possess either exactly one solution or infinitely many solutions.
  • If a non-trivial solution exists, the system possesses at least two solutions ($\mathbf{x} = \mathbf{0}$ and the non-trivial solution $\mathbf{x}$), which guarantees that the system has infinitely many solutions.

Questions and Discussion

• Conceptual Clarification on Span:

  • Question: How should the concept of "span" be understood within the context of linear systems?
  • Answer: The span represents the set of all possible linear combinations formed by a given set of vectors. Geometrically, it defines a subspace (such as a line or a plane passing through the origin). The span refers to the collection of vector outcomes generated by these linear combinations.

• Homework and Coursework Status:

  • Homework assignment coverage:
  • The homework assignment assigned on Friday is due on Monday / today.
  • Section 1.1 contains manageable foundational problems.
  • Section 1.2 contains 7 questions.
  • The full weekend is available to complete the remaining tasks.

• Scheduling and Personal Plans:

  • Work shift schedules mentioned: Working a 2:00 to 2:00 shift.
  • Weekend plans: Spending time with friends while managing approximately 20 pending tasks and commitments.