1.1 Systems of Linear Equations

1.1 Systems of Linear Equations

• Context and relevance

  • Linear programming, electrical networks, AI, signal processing, and machine learning rely on linear algebra and systems of linear equations.

  • This chapter introduces a systematic method (Gaussian elimination) for solving linear systems, defines matrix notation, and connects systems to vector and matrix equations.

• What is a linear equation?

  • A linear equation in variables x_1,x_2,\ldots,x_{n} has the form
    a_1x_1+a_2x_2+\cdots+a_{n}x_{n}=b

  • For example: 5x+3y=12, or -x_1+2x_2=5
    where the coefficients a_i and the constant term b are real or complex numbers.

  • An equation is linear if it can be rearranged into the form above; otherwise it is nonlinear (e.g., if a term involves \sqrt{x} or a product x_j).

• What is a system of linear equations?

  • A linear system is a collection of one or more linear equations in the same variables x_1, \dots, xn$.

  • Example system (two-equation, three-variable form):
    \begin{cases}
    a{11}x1 + a{12}x2 + a{13}x3 = b1, \ a{21}x1 + a{22}x2 + a{23}x3 = b2, \
    \vdots
    \end{cases}

  • A solution is a tuple $(x1, x2, \dots, x_n)$ that satisfies all equations simultaneously.

  • The solution set is the set of all possible solutions. Two systems are equivalent if they have the same solution set.

• Geometric interpretation (two equations in two variables)

  • Each equation represents a plane (in 3D, a line in 2D).

  • A solution is the intersection point of the planes (or lines in 2D).

  • Scenarios:

  • A unique intersection point (exactly one solution).

  • Parallel planes (no intersection; no solution).

  • Coincident planes (infinite intersections; infinitely many solutions).

• Possible sizes of a solution set (for a system in $\mathbb{R}^n$):

  • No solution (inconsistent).

  • Exactly one solution (unique).

  • Infinitely many solutions.

• Consistency

  • A system is consistent if it has at least one solution.

  • It is inconsistent if it has no solution.

• Matrix notation: coefficient and augmented matrices

  • Given the system
    x1 + 2x2 + x3 = 0, 2x1 + 8x2 + 3x3 = 8,
    {5x1 + ?x2 + 5x_3 = 10}

  • The coefficient matrix (matrix of coefficients) is
    A = \begin{bmatrix}
    a{11} & a{12} & a{13} \ a{21} & a{22} & a{23} \
    a{31} & a{32} & a_{33}
    \end{bmatrix}.

  • The augmented matrix is obtained by appending the right-hand side constants as an extra column:
    [A|b] = \begin{bmatrix}
    a{11} & a{12} & a{13} & | & b1 \
    a{21} & a{22} & a{23} & | & b2 \
    a{31} & a{32} & a{33} & | & b3
    \end{bmatrix}.

  • The size of a matrix is its dimensions $m \times n$ (rows $m$, columns $n$).

• Solving a linear system: the elimination strategy

  • Goal: replace the original system by an equivalent one that is easier to solve (same solution set).

  • Core idea: eliminate variables sequentially: use the $x1$-coefficient in the first equation to eliminate $x1$ from the other equations; then use the $x2$-coefficient in the second equation to eliminate $x2$ from the remaining equations, and so on.

• The three basic row operations (on the augmented matrix)

  • (1) Row replacement: Replace a row by the sum of itself and a multiple of another row:
    Ri \leftarrow Ri + c\,R_j, \quad i \neq j, c \in \mathbb{R}.

  • (2) Interchange: Swap two rows: Ri \leftrightarrow Rj.

  • (3) Scaling: Multiply a row by a nonzero constant: Ri \leftarrow c\,Ri, \quad c \neq 0.

  • These operations do not change the solution set of the corresponding system; they change only the representation.

  • Row operations are reversible; row equivalence means two matrices can be transformed into each other via a sequence of these operations.

  • Two systems with row-equivalent augmented matrices have the same solution set.

• Example 1: Solving a system via elimination (outline)

  • Start with a system in augmented form and perform elementary row operations to reach a triangular form (often called row echelon form).

  • Typical steps include:

  • Use the first equation to eliminate $x_1$ from the others.

  • Normalize rows to make leading coefficients equal to 1 (optional but convenient).

  • Use the new leading variables to eliminate above them in a back-substitution approach.

  • The text demonstrates that after a sequence of operations the system reduces to a triangular form like
    \begin{array}{ccc|c}
    x1 & \ast & \ast & 0 \ 0 & x2 & \ast & \ast \
    0 & 0 & x_3 & \ast
    \end{array}

  • Then back-substitution yields the solution; in the example, the original system has the solution $(x1,x2,x_3) = (1,0,-1)$, verified by substituting back into the original equations.

  • Key takeaway: row operations on the augmented matrix correspond to the same operations on the system of equations; the solution set remains unchanged.

• Existence and uniqueness questions (two fundamental questions)

  • 1) Is the system consistent (does at least one solution exist)?

  • 2) If a solution exists, is it unique (is the solution set a singleton)?)

  • These questions guide whether the system has no solution, a unique solution, or infinitely many solutions.

  • In practice, we answer these questions by row-reducing the augmented matrix and inspecting the final form.

• Example 2: Consistency check via row operations

  • Given the same or a related system, reduce to triangular form and inspect the last row.

  • If the last row reduces to something like 0\;0\;0\;|\;d\quad (d\neq 0), the system is inconsistent (no solution).

  • If the last row yields a consistent equation (e.g., 0=0 or a leading variable with a value), the system is consistent; the number of free variables determines whether the solution is unique or infinite.

  • In the example, the reduced triangular form shows that a solution exists and is unique for that system.

• Example 3: Inconsistency via a contradictory row

  • A system can be reduced to a form with a row like 0\;0\;0\;|\;15, which is impossible to satisfy, hence the system is inconsistent (no solution).

  • The augmented matrix’s last row exposes the contradiction directly.

• Checking solutions (sanity check)

  • Always substitute your candidate solution back into the original equations to verify correctness.

  • Example check: if a solution is found, substitute into each equation to confirm both sides are equal.

• Numerical note: floating-point arithmetic in practice

  • Computers usually solve square systems using elimination with floating-point arithmetic.

  • Real numbers are represented approximately as decimals with a finite number of digits, leading to roundoff errors.

  • Typical precision is 8–16 digits; inaccuracies are usually small but can be relevant in some problems.

  • The text warns about roundoff issues and notes where such concerns might arise in later chapters.

• Quick recap of key concepts

  • A linear system is a collection of linear equations in common variables.

  • The solution set can be none, one, or infinitely many solutions.

  • Consistency is about existence of at least one solution.

  • Row operations (replacement, interchange, scaling) preserve the solution set and allow us to transform the augmented matrix to a simpler form.

  • The augmented matrix compactly encodes the system; row-reducing it reveals the nature of the solutions.

  • Checking solutions by substitution is essential to ensure correctness.

• Practice problems (hint)

  • Before consulting solutions, attempt the practice problems related to identifying the next elementary row operation, determining consistency, and classifying the solution set.

• Summary of key notation

  • Linear system: a collection of equations in variables $x1, \dots, xn$.

  • Coefficient matrix: A = [a{ij}] consisting of the coefficients of $xj$ in each equation.

  • Augmented matrix: [A|b] includes the constants $b_i$ on the right.

  • Matrix size: an $m \times n$ matrix has $m$ rows and $n$ columns.

  • Row equivalence: a relation between matrices connected by a sequence of elementary row operations, preserving the solution set.

• Connections to later topics (brief)

  • The concepts of row operations and reduced forms lay the foundation for understanding spanning, linear independence, and linear transformations explored in later sections.

  • The idea of converting to an upper triangular (or row-echelon) form is a precursor to the notion of reduced row-echelon form used for systematic solutions.

• Key formulas to remember

  • General linear equation form:
    a1 x1 + a2 x2 + \cdots + an xn = b.

  • Coefficient matrix and augmented matrix structures:
    A = \begin{bmatrix} a{11} & a{12} & \cdots & a{1n} \\ \vdots & \vdots & \ddots & \vdots \\ a{m1} & a{m2} & \cdots & a{mn} \end{bmatrix}, \quad [A|b] = \begin{bmatrix} a{11} & a{12} & \cdots & a{1n} & | & b1 \\ \vdots & \vdots & \ddots & \vdots & | & \vdots \\ a{m1} & a{m2} & \cdots & a{mn} & | & bm \end{bmatrix}.

  • Elementary row operations (displayed succinctly):

  • $Ri \leftarrow Ri + c Rj$, $Ri \leftrightarrow Rj$, $Ri \leftarrow c R_i$ with $c \neq 0$.

• Final takeaway for exam prep

  • Be able to: (a) identify a linear system, (b) perform elementary row operations to reduce the augmented matrix, (c) determine consistency and uniqueness from the reduced form, and (d) verify a candidate solution by substitution.

• Title and scope

  • This set of notes covers Section 1.1: Systems of Linear Equations, including definitions, matrix notation, row operations, and core theorems about existence and uniqueness, with several worked examples and practical remarks.