Linear Equations in Linear Algebra Academic Study Notes

Linear Equations and Systems of Linear Equations

• A linear equation is an equation that can be written in the form:   $a_1 x_1 + a_2 x_2 + \dots + a_n x_n = b$   where $a_1, \dots, a_n$ and $b$ are real or complex numbers.

• The variables $x_1, \dots, x_n$ represent the unknowns of the linear equation.

• An equation is non-linear if any variable is modified by non-linear operations, such as being enclosed inside a square root (for example, $\sqrt{x_1}$).

• A system of linear equations (or linear system) is a collection of one or more linear equations involving the same set of variables $x_1, x_2, \dots, x_n$.

• A real-life application of linear systems includes determining unknown quantities when several simultaneous conditions must be satisfied, such as calculating the amounts of different ingredients in a mixture when total quantities of specific constituent elements are known.

• A solution of a system of linear equations is a set of specific values assigned to the variables $x_1, x_2, \dots, x_n$ that satisfies every equation in the system simultaneously.

• A linear system is classified according to the existence and number of its solutions:

  • Consistent system: A linear system that has either a unique solution or infinitely many solutions.

  • Inconsistent system: A linear system that has no solution.

Echelon Form and Row Reduced Echelon Form (RREF)

• A matrix is in Echelon Form if it satisfies the following three conditions:

  1. All non-zero rows are located above any all-zero rows (all zero rows are grouped at the bottom of the matrix).

  2. The leading entry (the first non-zero element from the left) of each non-zero row is in a column strictly to the right of the leading entry of the row directly above it.

  3. All entries in a column below a leading entry are equal to $0$

• A matrix is in Row Reduced Echelon Form (RREF) if it satisfies all three conditions of Echelon Form plus the following two additional conditions:

  1. The leading entry in every non-zero row is equal to $1$

  2. Each leading entry $1$ is the only non-zero entry in its column (all other entries in that column are $0$

• Key Definitions:

  • Pivot Element (or Pivot): The first non-zero entry in a non-zero row of a matrix in echelon form.

  • Pivot Column: A column in a matrix that contains a pivot element.

  • Basic Variable: A variable corresponding to a pivot column in the matrix.

  • Free Variable: A variable corresponding to a non-pivot column in the matrix.

Elementary Row Operations and Matrix Reduction

• Three elementary row operations can be performed on a matrix to transform it into echelon form or row reduced echelon form without altering the solution set of the corresponding linear system:

  1. Interchange: Swap the positions of any two rows $R_i$ and $R_j$ (denoted $R_i \leftrightarrow R_j$).

  2. Scaling: Multiply all entries of a row $R_i$ by a non-zero scalar $k$ (denoted $k R_i \rightarrow R_i$).

  3. Replacement: Replace a row $R_i$ with the sum of itself and a scalar multiple of another row $R_j$ (denoted $R_i + k R_j \rightarrow R_i$).

• Two matrices are row equivalent if one can be transformed into the other via a sequence of elementary row operations. Row equivalent augmented matrices represent linear systems with identical solution sets.

Step-by-Step Matrix Reduction Example

• Consider the matrix reduction process to obtain Echelon Form and Row Reduced Echelon Form:

• Initial Matrix:   $\begin{bmatrix} 0 & 1 & -4 & 8 \\ 2 & -3 & 2 & 1 \\ 5 & -8 & 7 & 1 \\ 3 & -4 & 1 & 5 \end{bmatrix}$

• Step 1: Interchange row $3$ and row $4$ ($R_3 \leftrightarrow R_4$):   $\begin{bmatrix} 0 & 1 & -4 & 8 \\ 2 & -3 & 2 & 1 \\ 3 & -4 & 1 & 5 \\ 5 & -8 & 7 & 1 \end{bmatrix}$

• Step 2: Interchange row $1$ and row $2$ ($R_1 \leftrightarrow R_2$):   $\begin{bmatrix} 2 & -3 & 2 & 1 \\ 0 & 1 & -4 & 8 \\ 3 & -4 & 1 & 5 \\ 5 & -8 & 7 & 1 \end{bmatrix}$

• Step 3: Eliminate lower entries in column $1$ using replacement operations $R_3 \rightarrow R_3 - \frac{3}{2} R_1$ and $R_4 \rightarrow R_4 - \frac{5}{2} R_1$:   $\begin{bmatrix} 2 & -3 & 2 & 1 \\ 0 & 1 & -4 & 8 \\ 0 & \frac{1}{2} & -2 & \frac{7}{2} \\ 0 & -\frac{1}{2} & 2 & -\frac{3}{2} \end{bmatrix}$

• Step 4: Eliminate lower entries in column $2$ using replacement operations $R_3 \rightarrow R_3 - \frac{1}{2} R_2$ and $R_4 \rightarrow R_4 + \frac{1}{2} R_2$:   $\begin{bmatrix} 2 & -3 & 2 & 1 \\ 0 & 1 & -4 & 8 \\ 0 & 0 & 0 & -\frac{1}{2} \\ 0 & 0 & 0 & \frac{5}{2} \end{bmatrix}$

• Step 5: Eliminate lower entries in column $4$ using replacement operation $R_4 \rightarrow R_4 + 5 R_3$:   $\begin{bmatrix} 2 & -3 & 2 & 1 \\ 0 & 1 & -4 & 8 \\ 0 & 0 & 0 & -\frac{1}{2} \\ 0 & 0 & 0 & 0 \end{bmatrix}$   This matrix is in Echelon Form.

• Step 6: Scale row $3$ by $-2$ ($-2 R_3 \rightarrow R_3$):   $\begin{bmatrix} 2 & -3 & 2 & 1 \\ 0 & 1 & -4 & 8 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix}$

• Step 7: Eliminate non-zero entries above the third pivot in column $4$ using $R_1 \rightarrow R_1 - R_3$ and $R_2 \rightarrow R_2 - 8 R_3$:   $\begin{bmatrix} 2 & -3 & 2 & 0 \\ 0 & 1 & -4 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix}$

• Step 8: Eliminate non-zero entries above the second pivot in column $2$ using $R_1 \rightarrow R_1 + 3 R_2$:   $\begin{bmatrix} 2 & 0 & -10 & 0 \\ 0 & 1 & -4 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix}$

• Step 9: Scale row $1$ by $\frac{1}{2}$ ($\frac{1}{2} R_1 \rightarrow R_1$):   $\begin{bmatrix} 1 & 0 & -5 & 0 \\ 0 & 1 & -4 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix}$   This matrix is in Row Reduced Echelon Form (RREF), where all pivot entries equal $1$ and all other entries in the pivot columns equal $0$

General Procedure for Solving Systems of Linear Equations

• Step 1: Construct the augmented matrix $[A \mid \mathbf{b}]$ representing the system of linear equations.

• Step 2: Perform elementary row operations (interchange, scaling, and replacement) to reduce the augmented matrix to echelon form.

• Step 3: Test for consistency:

  • If the rightmost column (the column of constants) contains a pivot position, the system contains an equation of the form $0 = b$ where $b \neq 0$, so no solution exists (inconsistent).

  • Otherwise, no pivot exists in the rightmost column, so a solution exists (consistent).

• Step 4: Continue row reduction until the matrix is in Row Reduced Echelon Form (RREF).

• Step 5: Write the simplified system of equations corresponding to the RREF matrix. Express basic variables in terms of any free variables to determine the general solution.

Systems of Linear Equations Solved Examples

• Example 1: System with a Unique Solution

  • Consider solving a linear system via augmented matrix reduction:     $\begin{bmatrix} 0 & 2 & -8 & 8 \\ 1 & -2 & 1 & 0 \\ -4 & 5 & 9 & -9 \end{bmatrix}$

  • Interchange row $1$ and row $2$ ($R_1 \leftrightarrow R_2$):     $\begin{bmatrix} 1 & -2 & 1 & 0 \\ 0 & 2 & -8 & 8 \\ -4 & 5 & 9 & -9 \end{bmatrix}$

  • Apply $R_3 \rightarrow R_3 + 4 R_1$:     $\begin{bmatrix} 1 & -2 & 1 & 0 \\ 0 & 2 & -8 & 8 \\ 0 & -3 & 13 & -9 \end{bmatrix}$

  • Scale row $2$ by $\frac{1}{2}$ ($\frac{1}{2} R_2 \rightarrow R_2$):     $\begin{bmatrix} 1 & -2 & 1 & 0 \\ 0 & 1 & -4 & 4 \\ 0 & -3 & 13 & -9 \end{bmatrix}$

  • Apply $R_3 \rightarrow R_3 + 3 R_2$:     $\begin{bmatrix} 1 & -2 & 1 & 0 \\ 0 & 1 & -4 & 4 \\ 0 & 0 & 1 & 3 \end{bmatrix}$

  • Eliminate entries above pivot $3$ ($R_2 \rightarrow R_2 + 4 R_3$ and $R_1 \rightarrow R_1 - R_3$):     $\begin{bmatrix} 1 & -2 & 0 & -3 \\ 0 & 1 & 0 & 16 \\ 0 & 0 & 1 & 3 \end{bmatrix}$

  • Eliminate entry above pivot $2$ ($R_1 \rightarrow R_1 + 2 R_2$):     $\begin{bmatrix} 1 & 0 & 0 & 48 \\ 0 & 1 & 0 & 16 \\ 0 & 0 & 1 & 3 \end{bmatrix}$

  • The system yields a unique solution:     $x_1 = 48$     $x_2 = 16$     $x_3 = 3$

• Example 2: System with Infinitely Many Solutions

  • Consider solving a linear system with augmented matrix:     $\begin{bmatrix} 1 & -2 & -1 & 3 \\ 3 & -6 & -2 & 2 \end{bmatrix}$

  • Apply $R_2 \rightarrow R_2 - 3 R_1$:     $\begin{bmatrix} 1 & -2 & -1 & 3 \\ 0 & 0 & 1 & -7 \end{bmatrix}$

  • Apply $R_1 \rightarrow R_1 + R_2$ to reach RREF:     $\begin{bmatrix} 1 & -2 & 0 & -4 \\ 0 & 0 & 1 & -7 \end{bmatrix}$

  • Corresponding linear equations:     $x_1 - 2x_2 = -4 \implies x_1 = 2x_2 - 4$     $x_3 = -7$

  • The variable $x_2$ is a free variable.

  • The general solution is:     $\begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} = \begin{pmatrix} 2x_2 - 4 \\ x_2 \\ -7 \end{pmatrix}$

  • Since $x_2$ can take any value, the system has infinitely many solutions.

• Example 3: Homogeneous System with Infinitely Many Solutions

  • Consider a homogeneous system $A \mathbf{x} = \mathbf{0}$ with 4 variables $x_1, x_2, x_3, x_4$

  • After performing row operations, the augmented matrix reduces to RREF:     $\begin{bmatrix} 1 & 0 & 0 & 2 & 0 \\ 0 & 1 & 0 & -3 & 0 \\ 0 & 0 & 1 & 1 & 0 \end{bmatrix}$

  • Columns $1$, $2$, and $3$ are pivot columns, corresponding to basic variables $x_1, x_2, x_3$

  • Column $4$ has no pivot, so $x_4$ is a free variable.

  • Corresponding equations:     $x_1 + 2x_4 = 0 \implies x_1 = -2x_4$     $x_2 - 3x_4 = 0 \implies x_2 = 3x_4$     $x_3 + x_4 = 0 \implies x_3 = -x_4$

  • Vector form of the general solution:     $\mathbf{x} = \begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \end{pmatrix} = \begin{pmatrix} -2x_4 \\ 3x_4 \\ -x_4 \\ x_4 \end{pmatrix} = x_4 \begin{pmatrix} -2 \\ 3 \\ -1 \\ 1 \end{pmatrix}$

  • Because $x_4$ is free, the homogeneous system has infinitely many solutions.

Homogeneous and Non-Homogeneous Linear Systems

• A system of linear equations written in matrix form as $A \mathbf{x} = \mathbf{b}$ is classified based on the vector of constants $\mathbf{b}$:

  • Homogeneous System: A system where $\mathbf{b} = \mathbf{0}$, giving $A \mathbf{x} = \mathbf{0}$

  • Non-Homogeneous System: A system where $\mathbf{b} \neq \mathbf{0}$

• Properties of Homogeneous Systems:

  • Every homogeneous system $A \mathbf{x} = \mathbf{0}$ is consistent because $\mathbf{x} = \mathbf{0}$ is always a solution. This solution is called the trivial solution.

  • A solution vector $\mathbf{x}$ with at least one non-zero component ($\mathbf{x} \neq \mathbf{0}$) is called a non-trivial solution.

  • A homogeneous system has a non-trivial solution if and only if the system has at least one free variable.

Application of Linear Systems: Balancing Chemical Reactions

• Systems of linear equations can be used to balance chemical reactions by ensuring atom conservation across all elements.

• General Balancing Method:

  1. Assign variables $x_1, x_2, \dots$ to represent the number of molecules of each reactant and product.

  2. Represent each molecule as a vector of atom counts for each element.

  3. Set up a vector equation equating reactants to products, forming a homogeneous linear system.

  4. Construct the augmented matrix and reduce it to RREF.

  5. Select the smallest positive integer for the free variable to yield whole-number stoichiometric coefficients.

• Reaction 1: Synthesis of Water ($H_2 + O_2 \rightarrow H_2O$)

  • Assign variables: $x_1 H_2 + x_2 O_2 \rightarrow x_3 H_2O$

  • Atom vectors ordered as $\begin{pmatrix} \text{Hydrogen} \\ \text{Oxygen} \end{pmatrix}$:

  • Hydrogen ($H_2$): $\begin{pmatrix} 2 \\ 0 \end{pmatrix}$

  • Oxygen ($O_2$): $\begin{pmatrix} 0 \\ 2 \end{pmatrix}$

  • Water ($H_2O$): $\begin{pmatrix} 2 \\ 1 \end{pmatrix}$

  • Vector Equation:     $x_1 \begin{pmatrix} 2 \\ 0 \end{pmatrix} + x_2 \begin{pmatrix} 0 \\ 2 \end{pmatrix} - x_3 \begin{pmatrix} 2 \\ 1 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}$

  • Augmented Matrix:     $\begin{bmatrix} 2 & 0 & -2 & 0 \\ 0 & 2 & -1 & 0 \end{bmatrix}$

  • Scale rows by $\frac{1}{2}$:     $\begin{bmatrix} 1 & 0 & -1 & 0 \\ 0 & 1 & -\frac{1}{2} & 0 \end{bmatrix}$

  • Equations:     $x_1 = x_3$     $x_2 = \frac{1}{2} x_3$

  • Choose $x_3 = 2$ to secure whole numbers, yielding $x_1 = 2$, $x_2 = 1$, $x_3 = 2$

  • Balanced Reaction:     $2 H_2 + O_2 \rightarrow 2 H_2 O$

• Reaction 2: Aluminum Oxidation ($Al + O_2 \rightarrow Al_2O_3$)

  • Assign variables: $x_1 Al + x_2 O_2 \rightarrow x_3 Al_2 O_3$

  • Atom vectors ordered as $\begin{pmatrix} \text{Aluminum} \\ \text{Oxygen} \end{pmatrix}$:

  • Aluminum ($Al$): $\begin{pmatrix} 1 \\ 0 \end{pmatrix}$

  • Oxygen ($O_2$): $\begin{pmatrix} 0 \\ 2 \end{pmatrix}$

  • Aluminum Oxide ($Al_2 O_3$): $\begin{pmatrix} 2 \\ 3 \end{pmatrix}$

  • Vector Equation:     $x_1 \begin{pmatrix} 1 \\ 0 \end{pmatrix} + x_2 \begin{pmatrix} 0 \\ 2 \end{pmatrix} - x_3 \begin{pmatrix} 2 \\ 3 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}$

  • Augmented Matrix:     $\begin{bmatrix} 1 & 0 & -2 & 0 \\ 0 & 2 & -3 & 0 \end{bmatrix}$

  • Scale row 2 by $\frac{1}{2}$:     $\begin{bmatrix} 1 & 0 & -2 & 0 \\ 0 & 1 & -\frac{3}{2} & 0 \end{bmatrix}$

  • Equations:     $x_1 = 2 x_3$     $x_2 = \frac{3}{2} x_3$

  • Choose $x_3 = 2$ to avoid fractional coefficients, yielding $x_1 = 4$, $x_2 = 3$, $x_3 = 2$

  • Balanced Reaction:     $4 Al + 3 O_2 \rightarrow 2 Al_2 O_3$

Linearly Dependent and Independent Vectors

• Linearly Dependent Vectors:

  • A set of vectors $\{\mathbf{v_1}, \mathbf{v_2}, \dots, \mathbf{v_n}\}$ in a vector space $V$ is linearly dependent if their linear combination can equal zero while at least one scalar $c_i$ is non-zero ($c_i \neq 0$):     $c_1 \mathbf{v_1} + c_2 \mathbf{v_2} + \dots + c_n \mathbf{v_n} = \mathbf{0}$

  • Equivalently, a set is linearly dependent if there exists a non-trivial solution to the vector equation $c_1 \mathbf{v_1} + \dots + c_n \mathbf{v_n} = \mathbf{0}$

• Linearly Independent Vectors:

  • A set of vectors $\{\mathbf{v_1}, \mathbf{v_2}, \dots, \mathbf{v_n}\}$ in a vector space $V$ is linearly independent if their linear combination equals zero only when every scalar is zero:     $c_1 \mathbf{v_1} + c_2 \mathbf{v_2} + \dots + c_n \mathbf{v_n} = \mathbf{0} \implies c_1 = c_2 = \dots = c_n = 0$

• Key Principles of Linear Dependence and Independence:

  • A set consisting of a single zero vector $\{\mathbf{0}\}$ is always linearly dependent.

  • A set containing two or more vectors is linearly dependent if and only if at least one vector in the set can be expressed as a linear combination of the remaining vectors.

  • Introducing (including) a zero vector into any set of vectors makes the entire set linearly dependent.

  • The central test for linear dependence or independence is determining whether $c_1 \mathbf{v_1} + \dots + c_n \mathbf{v_n} = \mathbf{0}$ permits a non-trivial solution.

• Examples:

  • Example 1: Consider vectors $\mathbf{v_1} = \begin{pmatrix} 1 \\ 2 \end{pmatrix}$ and $\mathbf{v_2} = \begin{pmatrix} 2 \\ 4 \end{pmatrix}$. Setting up the combination:     $2 \mathbf{v_1} - \mathbf{v_2} = 2 \begin{pmatrix} 1 \\ 2 \end{pmatrix} - \begin{pmatrix} 2 \\ 4 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}$     Since non-zero scalars $c_1 = 2$ and $c_2 = -1$ produce the zero vector, the vectors are linearly dependent.

  • Example 2: Consider a set containing a zero vector, such as $\{\mathbf{v_1}, \mathbf{v_2}, \mathbf{0}\}$. Choosing $c_1 = 0$, $c_2 = 0$, and $c_3 = 1 \neq 0$ gives:     $0 \mathbf{v_1} + 0 \mathbf{v_2} + 1 \mathbf{0} = \mathbf{0}$     The zero vector itself allows a non-trivial coefficient choice, making the set linearly dependent.