The Matrix Equation Ax = b - Comprehensive Study Notes

Formal Definition: If $A$ is an $m \times n$ matrix, with columns $\mathbf{a}_1, \mathbf{a}_2, \dots, \mathbf{a}_n$ , and if $\mathbf{x}$ is in $\mathbb{R}^n$ , then the product of $A$ and $\mathbf{x}$ , denoted by $A\mathbf{x}$ , is the linear combination of the columns of $A$ using the corresponding entries in $\mathbf{x}$ as weights.
Mathematical Expression: $A\mathbf{x} = [\mathbf{a}_1 \ \mathbf{a}_2 \ \dots \ \mathbf{a}_n] \begin{pmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{pmatrix} = x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \dots + x_n\mathbf{a}_n$
Existence Constraint: The product $A\mathbf{x}$ is defined only if the number of columns of the matrix $A$ is equal to the number of entries in the vector $\mathbf{x}$ .

Example 1: Writing a linear combination as a matrix-vector product.
- Problem: For vectors $\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3$ , write the linear combination $4\mathbf{v}_1 - 4\mathbf{v}_2 + 7\mathbf{v}_3$ as a matrix times a vector.
- Solution: Place the vectors $\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3$ into the columns of a matrix $A$ and place the weights $4, -4,$ and $7$ into a vector $\mathbf{x}$ .
- Result: $[\mathbf{v}_1 \ \mathbf{v}_2 \ \mathbf{v}_3] \begin{pmatrix} 4 \\ -4 \\ 7 \end{pmatrix}$

System Transformation: A system of linear equations can be represented as a vector equation involving a linear combination of vectors.
- Example: The system: $\begin{cases} x_1 + 2x_2 - x_3 = 4 \\ -5x_2 + 3x_3 = 1 \end{cases}$
- Is equivalent to the vector equation: $x_1 \begin{pmatrix} 1 \\ 0 \end{pmatrix} + x_2 \begin{pmatrix} 2 \\ -5 \end{pmatrix} + x_3 \begin{pmatrix} -1 \\ 3 \end{pmatrix} = \begin{pmatrix} 4 \\ 1 \end{pmatrix}$
- Which is equivalent to the matrix equation ( $A\mathbf{x} = \mathbf{b}$ ): $\begin{pmatrix} 1 & 2 & -1 \\ 0 & -5 & 3 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} = \begin{pmatrix} 4 \\ 1 \end{pmatrix}$
Theorem 3: If $A$ is an $m \times n$ matrix, with columns $\mathbf{a}_1, \dots, \mathbf{a}_n$ , and if $\mathbf{b}$ is in $\mathbb{R}^m$ , the matrix equation $A\mathbf{x} = \mathbf{b}$ has the same solution set as the vector equation: $x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \dots + x_n\mathbf{a}_n = \mathbf{b}$ which, in turn, has the same solution set as the system of linear equations whose augmented matrix is: $[\mathbf{a}_1 \ \mathbf{a}_2 \ \dots \ \mathbf{a}_n \ \mathbf{b}]$

Fundamental Criterion: The equation $A\mathbf{x} = \mathbf{b}$ has a solution if and only if $\mathbf{b}$ is a linear combination of the columns of $A$ .
Theorem 4 (Equivalence Theorem): Let $A$ be an $m \times n$ matrix. The following four statements are logically equivalent (for a specific $A$ , they are either all true or all false):
- (a) For each $\mathbf{b}$ in $\mathbb{R}^m$ , the equation $A\mathbf{x} = \mathbf{b}$ has a solution.
- (b) Each $\mathbf{b}$ in $\mathbb{R}^m$ is a linear combination of the columns of $A$ .
- (c) The columns of $A$ span $\mathbb{R}^m$ .
- (d) $A$ has a pivot position in every row.

Strategy: Since (a), (b), and (c) are logically equivalent by definition, it suffices to show that (a) and (d) are either both true or both false for an arbitrary matrix $A$ .
Process:
- Let $U$ be an echelon form of $A$ .
- For any $\mathbf{b}$ in $\mathbb{R}^m$ , the augmented matrix $[A \ \mathbf{b}]$ can be row reduced to $[U \ \mathbf{d}]$ for some vector $\mathbf{d}$ .
Case 1 (d is true): If statement (d) is true, then each row of $U$ contains a pivot position. Consequently, there can be no pivot in the augmented column of the reduced matrix. Thus, $A\mathbf{x} = \mathbf{b}$ has a solution for any $\mathbf{b}$ , making (a) true.
Case 2 (d is false): If (d) is false, the last row of $U$ is all zeros. If we choose a vector $\mathbf{d}$ where the last entry is $1$ , then the system $[U \ \mathbf{d}]$ is inconsistent. Because row operations are reversible, this can be transformed back to an original system $[A \ \mathbf{b}]$ which is also inconsistent. Therefore, statement (a) is false.

Example 2 (Definition Method): Compute $A\mathbf{x}$ , where: $A = \begin{pmatrix} 2 & 3 & 4 \\ -1 & 5 & -3 \end{pmatrix}, \mathbf{x} = \begin{pmatrix} 1 \\ -2 \\ 1 \end{pmatrix}$ Using the definition: $A\mathbf{x} = 1 \begin{pmatrix} 2 \\ -1 \end{pmatrix} - 2 \begin{pmatrix} 3 \\ 5 \end{pmatrix} + 1 \begin{pmatrix} 4 \\ -3 \end{pmatrix} = \begin{pmatrix} 2 - 6 + 4 \\ -1 - 10 - 3 \end{pmatrix} = \begin{pmatrix} 0 \\ -14 \end{pmatrix}$
Row-Vector Rule for Computing Ax:
- The first entry in $A\mathbf{x}$ is the sum of products (a dot product) using the first row of $A$ and the entries in $\mathbf{x}$ .
- Specifically, the entry in row $i$ of the resulting vector $A\mathbf{x}$ is calculated by multiplying the entries in the $i$ -th row of $A$ by the corresponding entries in $\mathbf{x}$ and summing the results.
- Calculation for Example 2 using the Row-Vector Rule:
  - Entry 1: $(2 \times 1) + (3 \times -2) + (4 \times 1) = 2 - 6 + 4 = 0$
  - Entry 2: $(-1 \times 1) + (5 \times -2) + (-3 \times 1) = -1 - 10 - 3 = -14$

Definition: The matrix with $1$ s on the main diagonal (from top-left to bottom-right) and $0$ s elsewhere is called the identity matrix and is denoted by $I$ .
Notation: If the size needs to be specified, it is written as $I_n$ for an $n \times n$ identity matrix.
Property: For any $\mathbf{x}$ in $\mathbb{R}^n$ , $I\mathbf{x} = \mathbf{x}$ .
Example ( $3 \times 3$ Identity Matrix): $I = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}$

Theorem 5: Let $A$ be an $m \times n$ matrix, let $\mathbf{u}$ and $\mathbf{v}$ be vectors in $\mathbb{R}^n$ , and let $c$ be a scalar. Then:
- (a) $A(\mathbf{u} + \mathbf{v}) = A\mathbf{u} + A\mathbf{v}$
- (b) $A(c\mathbf{u}) = c(A\mathbf{u})$
Proof Logic:
- For simplicity, assume $n=3$ and the columns of $A$ are $\mathbf{a}_1, \mathbf{a}_2, \mathbf{a}_3$ . Let $u_i$ and $v_i$ be the entries of $\mathbf{u}$ and $\mathbf{v}$ .
- Proving (a): Compute $A(\mathbf{u} + \mathbf{v})$ as a linear combination of columns: $(u_1 + v_1)\mathbf{a}_1 + (u_2 + v_2)\mathbf{a}_2 + (u_3 + v_3)\mathbf{a}_3$ By regrouping terms: $(u_1\mathbf{a}_1 + u_2\mathbf{a}_2 + u_3\mathbf{a}_3) + (v_1\mathbf{a}_1 + v_2\mathbf{a}_2 + v_3\mathbf{a}_3) = A\mathbf{u} + A\mathbf{v}$
- Proving (b): Compute $A(c\mathbf{u})$ as a linear combination using entries $cu_i$ as weights: $(cu_1)\mathbf{a}_1 + (cu_2)\mathbf{a}_2 + (cu_3)\mathbf{a}_3 = c(u_1\mathbf{a}_1 + u_2\mathbf{a}_2 + u_3\mathbf{a}_3) = c(A\mathbf{u})$

This material is from "Linear Algebra and Its Applications" Sixth Edition by Pearson Education, Inc.
This work is protected by United States copyright laws.
The material is provided for instructor use in teaching and assessing student learning; sale or dissemination (including on the web) is not permitted.