Notes on Systems of Linear Equations and Matrix Methods

System of Linear Equations: Overview

• A system of linear equations is a collection of one or more equations involving the same variables.

• Goal: find values of the variables that satisfy all equations simultaneously.

• Common notations: variables are typically x1, x2, x3, … ; coefficients form a matrix; right-hand sides form a column vector.

Three Forms of Representing a System

• Linear equations form:

  • Example (three variables):
    x1 - 3x2 + 4x_3 = -4

• Vector equation form:

  • Writing the system as a vector equation with a matrix acting on a variable vector:

  • If A is the coefficient matrix and x is the column vector of unknowns, then the system can be written as
    A\,\mathbf{x} = \mathbf{b}

  • Example structure: A is a matrix with rows corresponding to equations and columns to variables; x is a vector \mathbf{x} = \begin{bmatrix}x1 \ x2 \ x3\end{bmatrix}; b is the constants vector \mathbf{b} = \begin{bmatrix}b1 \ b2 \ b3\end{bmatrix}

• Matrix equation form (augmented with the constant column):

  • Often written with an augmented matrix [A|b], representing the same system.

• Quick examples:

  • For the system in vector form, the relation is conceptually: given A and b, solve for x in A\,\mathbf{x} = \mathbf{b}.

  • A concrete 3×3 system might look like
    \begin{bmatrix}1 & -2 & 2\ 2 & 1 & -1\ 3 & 0 & 4\end{bmatrix} \begin{bmatrix}x1\ x2\ x_3\end{bmatrix} = \begin{bmatrix}7\ 1\ 5\end{bmatrix}.n

Solving Methods (Overview)

• Substitution method

• Elimination method (Triangle Form / Gaussian Elimination)

• Echelon Form (Row Echelon Form, REF)

• Reduced Echelon Form (Row Reduced Echelon Form, RREF) / Gauss-Jordan Elimination

• Cramer’s Rule (for square systems with nonzero determinant)

• Sweep-out method

• Using matrix inverse (A^{-1} b) for A invertible

Substitution Method

• Idea: solve one equation for one variable; substitute into others.

• Worked example (two equations in two variables):

  • System:
    \begin{cases} x + 2y = 10 \\ 0.5x - 3y = -11 \end{cases}

  • From the first equation: x = 10 - 2y

  • Substitute into the second:
    0.5(10 - 2y) - 3y = -11

  • Compute: 5 - y - 3y = -11 \Rightarrow 5 - 4y = -11

  • Solve: -4y = -16 \Rightarrow y = 4

  • Back-substitute: x = 10 - 2\cdot 4 = 2

  • Solution: (x, y) = (2, 4)

• Notes:

  • Simple and intuitive for small systems.

  • Can become tedious for larger systems with many substitutions.

Elimination Method (Triangular Form) / Gaussian Elimination

• Idea: use row operations to transform the augmented matrix into an upper-triangular (triangular) form, then back-substitute.

• Allowed row operations (operations preserve the solution set):

  • Swap two rows: Ri <-> Rj

  • Multiply a row by a nonzero constant: k Ri -> Ri (k ≠ 0)

  • Replace a row by the sum of itself and a multiple of another row: Ri <- Ri + c R_j

• Example system (in augmented matrix form):

  • System:
    \begin{cases} x1 - 3x2 + 4x3 = -4 \\ 3x1 - 7x2 + 7x3 = -8 \\ -4x1 + 6x2 - x_3 = 7 \end{cases}

  • Augmented matrix:
    \left[\begin{array}{ccc|c}
    1 & -3 & 4 & -4 \
    3 & -7 & 7 & -8 \
    -4 & 6 & -1 & 7
    \end{array}\right]

  • Row operations lead to a row of the form [0\ 0\ 0\ |\ 3] or similar, indicating inconsistency (0 = 3), hence no solution.

• Outcome:

  • If you reach a row with all zero coefficients but a nonzero constant on the right, the system is inconsistent (no solution).

  • If you reach an upper-triangular form and can back-substitute, you obtain a solution.

Echelon Form (Row Echelon Form, REF)

• Definition/characteristics:

  • All nonzero rows are above any rows of all zeros.

  • The leading (leftmost nonzero) entry of a row is to the right of the leading entry of the row above it.

  • All entries below a leading entry are zero.

• pivot and pivot columns:

  • A pivot is the leading entry (the first nonzero in that row).

  • The column containing a pivot is a pivot column.

• Purpose:

  • REF is a simpler form than the original, enabling easier solving and analysis of rank.

Reduced Row Echelon Form (RREF) / Gauss-Jordan Elimination

• Goals/definition:

  • In addition to REF properties, every pivot is 1 and is the only nonzero entry in its column (i.e., zeros above and below each pivot).

• Operations (same three rules above) used to reach RREF:

  • Continue row operations until each pivot column has a single 1 and zeros elsewhere.

• Properties:

  • The RREF of a given matrix is unique (there is exactly one RREF for a given matrix).

  • The RREF directly reveals the solution structure of the system.

• Notation:

  • Pivot positions correspond to leading 1s in the RREF.

  • Free variables correspond to columns without pivots.

Pivot Columns and Solutions

• If the coefficient matrix A has a pivot in every column (i.e., rank(A) = n for an n-variable system), then there is a unique solution.

• If there are fewer pivots than variables (rank(A) < n), there are free variables and infinitely many solutions (when the system is consistent).

• Inconsistent systems are detected when a row reduces to [0\ 0\ \dots\ 0 \mid b] with b \neq 0.

Gaussian vs Gauss-Jordan Elimination (Workflow)

• Gaussian elimination (to REF) then back-substitution to solve for the variables.

• Gauss-Jordan elimination (to RREF) directly gives the solution by inspecting the pivot positions and free variables.

Uniqueness of the Reduced Echelon Form

• Fact: For any given matrix, there is exactly one RREF.

• This property underpins the interpretation of the row-reduction outcome in terms of solution sets.

Describing the Solutions from RREF

• Inconsistent: at least one row is [0\ \cdots \ 0\mid b] with b \neq 0.

• Consistent:

  • Unique solution: rank(A) = n and augmented rank = n; all variables determined (no free variables).

  • Infinitely many solutions: rank(A) = r < n; there are (n - r) free variables; solution set is an affine subspace parameterized by the free variables.

Gauss-Jordan Elimination: Example Outline

• Given a system, form the augmented matrix and apply row operations to reach RREF.

• If you end with a unique pivot in each variable column and no inconsistent row, you read off the solution from the right-hand side.

• If a variable is free, express dependent variables in terms of the free parameter(s).

Example: Unique Solution via Gauss-Jordan

• Example system (three variables) with a successful RREF giving a unique solution:

  • After reduction, the augmented form corresponds to a pivot in every coefficient column, leading to a unique solution, for instance
    (x1, x2, x_3) = (1, 3, 4).

• This mirrors the transcript where one example concluded the solution is (1, 3, 4) .

Example: Inconsistent System via Elimination

• A system reduced to a row with all zero coefficients but a nonzero constant on the right, e.g.
  [0\ 0\ 0\mid 3],
  indicates no solution (inconsistent).

Example: Infinite Solutions (Free Variable)

• A system with at least one free variable yields infinitely many solutions.

• Example form from the transcript (parameterization):

  • Suppose after reduction we have
    x1 + x3 = 0,
    2x2 - 3x3 = 1,
    x_3 ext{ free}.

  • Let x_3 = t (t ∈ ℝ).

  • Then x1 = -t, or 2x2 = 1 + 3t \Rightarrow x_2 = \frac{1+3t}{2}.

  • General solution: (x1, x2, x_3) = (-t, \tfrac{1+3t}{2}, t) for all real t.

• The transcript shows a similar pattern where a variable (x3) is free and other variables are expressed in terms of x3 (or t).

Summary of Practical Workflow for Solving Systems

• Step 1: Write the augmented matrix [A|b] for the system A x = b.

• Step 2: Apply row operations to reduce to REF or RREF (via Gaussian or Gauss-Jordan elimination).

• Step 3: Check for inconsistency (a row [0 … 0 | b] with b ≠ 0).

• Step 4: If consistent, determine pivots and free variables.

• Step 5: Read off the solution: if all variables are leading (no free vars) → unique solution; if some variables are free → infinite solutions parameterized by free variables.

• Step 6: If convenient, express the solution in vector form or parametric form.

Quick Reference: Notation and Concepts

• Matrix notation:

  • Coefficient matrix: A = [a_{ij}] with size m×n.

  • Variable vector: \mathbf{x} = \begin{bmatrix}x1 \ x2 \ \vdots \ x_n\end{bmatrix}

  • Right-hand side vector: \mathbf{b} = \begin{bmatrix}b1 \ b2 \ \vdots \ b_m\end{bmatrix}

• Pivot: the first nonzero entry in a row; pivot position marks the leading 1 in RREF.

• Pivot column: a column that contains a pivot.

• Rank: number of pivots in the coefficient matrix (or the augmented matrix, if considering consistency).

• Invertible A: if and only if det(A) ≠ 0; then the solution to A x = b is x = A^{-1} b (when A is square and b is compatible).

Practice and Exercises (From Transcript Context)

• The transcript includes multiple worked examples and practice problems aimed at solving systems via reduced echelon form (RREF).

• Practice direction: solve the given systems using RREF; interpret whether the system is inconsistent, has a unique solution, or has infinitely many solutions (expressed with parameters for the free variables).

Quick Takeaways

• Substitution and elimination are foundational methods; for larger systems, matrix methods (REF/RREF, Gauss-Jordan) are efficient.

• RREF provides a unique canonical form for any matrix, directly revealing the solution structure.

• The presence of free variables corresponds to infinitely many solutions; absence of free variables corresponds to a unique solution; any inconsistency in a row of the augmented matrix yields no solution.

• Matrix inverses offer another route for square, invertible A: solve by x = A^{-1} b.