Linear Systems, Matrix Algebra, and Analytic Geometry
Systems of Linear Equations
Definition of a Linear Equation:
A linear equation in variables has the general standard form:
The numbers are constants called the coefficients of the variables, and is a constant called the constant term.
For two variables, this corresponds to the line in : .
For three variables, this corresponds to the plane in : .
Classifying Equations as Linear or Non-Linear:
: Linear. All variables are to the first power with constant coefficients.
: Non-linear. Contains a product of variables ().
: Non-linear. Involves division by a variable ().
: Linear. Constant coefficients with variables to the first power.
: Non-linear. Variables are squared ().
: Linear. Real decimal coefficients, degree-one variables.
: Linear. Rearranges to .
: Linear. The coefficient is a constant scalar multiplying .
: Non-linear. The variable is inside a non-linear trigonometric function
Solutions to Linear Equations:
A column vector is a solution to if substitution yields the true scalar identity:
For :
Setting gives , yielding solution .
Setting gives , yielding solution .
Geometrically, the full solution set represents a straight line in .
Systems of Linear Equations:
A system of linear equations is a finite set of linear equations involving the same set of variables.
A solution to a system is a vector that simultaneously satisfies every equation in the system.
Two systems are defined as equivalent if and only if they possess the exact same solution set.
Fundamental Outcome Theorem for Real Linear Systems:
Every real linear system has exactly one of the following three outcomes:
Exactly one unique solution.
Infinitely many solutions.
No solution.
A system is termed consistent if it possesses at least one solution (outcomes 1 or 2).
A system is termed inconsistent if it possesses no solution (outcome 3).
Geometric Examples of System Outcomes:
Unique Solution: Adding both equations yields , so . The solution is the single point . Geometrically, this represents two non-parallel lines intersecting at a single point.
Infinitely Many Solutions: The second equation is a multiple of the first ( first equation). Any point on satisfies both equations. Geometrically, these equations represent coincident lines.
No Solution: Subtracting the equations yields , a contradiction. Geometrically, these equations represent parallel and distinct lines in .
Solving Systems by Back Substitution:
A system in triangular form can be solved directly from bottom to top.
Consider the system:
Solve the bottom equation: .
Substitute into the middle equation: .
Substitute and into the top equation: .
The unique solution vector is .
Matrix Representation of Systems:
For a system :
The coefficient matrix contains exclusively the coefficients of the variables.
The augmented matrix contains the coefficient columns augmented by the constant column .
Example conversion:
Coefficient matrix:
Augmented matrix:
Row Echelon Form (REF):
A matrix is in Row Echelon Form (REF) if:
All all-zero rows are grouped together at the bottom of the matrix.
In each non-zero row, the first non-zero entry (the leading entry or pivot) lies strictly to the right of the leading entry of the row directly above it.
This creates a characteristic staircase pattern.
Valid REF matrices:
Elementary Row Operations:
Elementary row operations transform an augmented matrix into an equivalent augmented matrix without changing the solution set:
Interchange two rows:
Multiply a row by a non-zero scalar:
Add a scalar multiple of one row to another row:
Gaussian Elimination and Gauss-Jordan Elimination:
Gaussian Elimination: Transform the augmented matrix to REF using elementary row operations, then extract the solution using back substitution.
Reduced Row Echelon Form (RREF): A matrix is in RREF if:
It is in REF.
Every leading entry in a non-zero row is equal to (a leading 1).
Every column containing a leading 1 has zeros in all other positions (both above and below).
Gauss-Jordan Elimination: Transform the augmented matrix to REF, verify consistency, and continue row operations upward until the matrix reaches RREF.
Worked Row Reduction (Example 5 & Example 5 Redux):
Consider the augmented matrix:
Elimination to REF:
Perform and :
Perform :
The matrix is now in REF.
Continuation to RREF:
Clear above the leading 1 in row 2: perform :
Parametric Solution:
Leading variables (corresponding to columns with leading 1s): .
Free variables: .
Row equations: ; .
Vector form:
Rank and the Rank Theorem:
Definition of Rank: The rank of a matrix , denoted , is the total number of leading entries (pivots) present after is reduced to Row Echelon Form.
Rank Theorem (Version 1): For a consistent linear system having coefficient matrix with variables:
Parameter and Shape Analysis Table:
System 1: Variables: . Leading entries: . Parameters: . Solutions: unique solution. Shape: Single point.
System 2: Variables: . Leading entries: (). Parameters: . Solutions: Infinitely many. Shape: Line in .
System 3: Variables: . Leading entries: (). Parameters: . Solutions: Infinitely many. Shape: Two-dimensional plane in .
Example Analysis without Explicit Solving: Number of variables: . Rank of coefficient matrix: . Since it is consistent, number of parameters is . The geometric shape of the solution set is a straight line in .
Homogeneous Systems:
A linear system is defined as homogeneous if all constant terms are zero:
Every homogeneous system is guaranteed to have at least the trivial solution:
A homogeneous system can never be inconsistent (it always has at least one solution).
Underdetermined Homogeneous Systems Theorem: If a homogeneous system has equations and variables with fewer equations than variables (), the system is guaranteed to possess infinitely many solutions.
Justification: . Therefore, the number of free parameters . At least one free parameter exists.
Matrix Algebra and Special Matrices
Definition of a Matrix:
An matrix is a rectangular array of numbers arranged into horizontal rows and vertical columns:
The scalar represents the entry located in row and column .
A matrix can also be expressed compactly by writing its column vectors:
The entries form the main diagonal of the matrix (where the row index equals the column index).
Notation Check:
For the matrix:
Size: ( rows, columns).
Entry (row 1, column 2).
Entry (row 2, column 1).
Second column: .
Special Matrix Classes:
Zero Matrix (): Every entry equals . Its dimensions depend on context (e.g., ).
Diagonal Matrix: A square matrix where all entries outside the main diagonal are zero ( for all ), such as .
Scalar Matrix: A diagonal matrix whose main diagonal entries are all identical, equal to (e.g., ).
Identity Matrix (): A square matrix with s along the main diagonal and s everywhere else:
Matrix Addition and Scalar Multiplication:
Matrices must have identical dimensions to be added or subtracted.
If and are both , then:
Subtraction is defined by .
Algebraic Rules for Addition and Scaling:
For same-size matrices and real scalars :
Commutativity of Addition:
Additive Identity:
Additive Inverse:
Associativity of Addition:
Distributive over Matrix Addition:
Distributive over Scalar Addition:
Scalar Associativity:
Identity Scaling:
Matrix Algebraic Computations:
Combining Like Terms:
Solving Linear Matrix Equations:
Linear Combinations and Span of Matrices:
If have the same dimensions, a linear combination is:
The span of a set of matrices is the set of all possible linear combinations that can be formed from them.
Given:
General combination:
Crucial structural property: The two diagonal entries are strictly equal ().
Testing :
Diagonal entries match (), so .
Off-diagonal: and .
Adding off-diagonal equations gives .
Then , and .
Since consistent, . Thus, .
Testing :
Top-left entry requires .
Bottom-right entry requires .
Since a scalar cannot simultaneously equal and , this system is inconsistent. Thus, .
Description of Full Span: For any real values , the unique combination coefficients are:
Linear Independence of the Matrices:
Setting the linear combination to the zero matrix:
System: , , .
Adding the first two equations gives , which forces and .
Only the trivial solution exists; therefore, the set of matrices is linearly independent.
Matrix Operations and Transpose Identities
Definition of Matrix Product:
Let be an matrix and be a matrix. The product is an matrix whose entry is given by:
Entry is the dot product of row of matrix with column of matrix .
Multiplication requirement: The inner dimensions must match (). The outer dimensions () dictate the dimensions of the product matrix:
Properties of Matrix Multiplication:
For dimensionally compatible matrices and scalar :
Associative Law:
Left Distributive Law:
Right Distributive Law:
Scalar Compatibility:
Identity Multiplicative Property: For an matrix , .
Failures of Real Number Arithmetic in Matrix Multiplication:
Non-Commutativity: In general, , even when both products are defined and have the same dimensions.
Zero Divisors (Cancellation Failure): For real numbers, . For matrices, does not imply or .
Explicit counterexample: Both and are non-zero, yet . Furthermore, .
Matrix Powers:
Matrix powers are defined only for square matrices (). For integer :
Exponent rule: for non-negative integers .
Example Cycle of Powers:
Let .
The powers cycle with period 4: .
Expansion of Binomial Products:
For square matrices and of the same size:
The familiar elementary algebraic formula holds if and only if (that is, and commute).
Block Matrix Multiplication:
Large matrices can be partitioned into submatrices (blocks).
When the column partitions of the first matrix match the row partitions of the second matrix, block multiplication proceeds identically to regular matrix multiplication, treating submatrices as elements.
Zero and identity blocks within partitions dramatically accelerate computations.
Matrix Transpose:
For an matrix , the transpose is the matrix produced by interchanging the rows and columns:
An entry at row , column transitions to row , column .
Examples:
For of size , its transpose is size . Entry at position relocates to position .
Transpose Algebraic Identities:
For compatible matrices , scalar , and integer :
Transpose of a Transpose:
Transpose of a Sum:
Transpose of a Scalar Multiple:
Transpose of a Product (Order Reversal):
Transpose of a Power:
Transpose of a Triple Product: .
Entry-Level Proof of the Product Transpose Rule :
Let be and be . Then is , and is .
The entry of is the entry of :
Now consider the matrix product . Matrix is and is , so their product is .
The entry of is:
Because real number multiplication is commutative (), the entries are identical for all . Thus, .
Dot Product and Vector Norm via Transpose:
For column vectors :
Dot Product:
Dimensions: .
Example: For and :
Squared Norm:
Norm (Length):
Example: For , .
Extracting Rows and Columns with Standard Basis Vectors:
Let denote the -th standard basis vector (having in entry and elsewhere):
Row Selection: selects row of (where ).
In , with :
Column Selection: selects column of (where ).
In , with :
Vector Equations of Lines and Planes
Lines in :
A point and a non-zero direction vector determine a unique line :
Interpretation: The vector locates the starting point , the vector defines the line's orientation in space, and the parameter dictates the scalar multiple of added to .
Worked Example:
Parametric Equations:
At , the point is .
At , the point reached is .
For any point to lie on the line, all coordinates must be produced simultaneously by the exact same parameter value .
Multiple Descriptions of the Same Line:
A line possesses infinitely many valid vector equations.
Any point lying on the line can serve as the base point .
The direction vector may be replaced by any non-zero scalar multiple ().
A negative scalar multiple () reverses the directional orientation of traversal along the line, but leaves the physical set of points comprising the line unchanged.
Planes in :
For a point and two non-zero, non-parallel vectors , a two-dimensional plane is defined by:
This parametric definition describes a two-dimensional flat surface regardless of the ambient dimension of the space.
Worked Example in (Example 18):
Point , directions , .
Vector Equation:
Parametric Equations:
Dimensional Comparison Table:
Line in : Requires non-zero direction vector; uses parameter ().
Plane in : Requires non-zero, non-parallel direction vectors; uses parameters ().
Note on ambient dimension: A plane in requires four coordinates to specify position, but retains exactly two internal degrees of freedom (independent parameters). Cross product methods are strictly restricted to .
Classification of Lines in Three-Dimensional Space
The Four Geometric Relationships for Two Lines in :
Intersecting: The lines possess non-parallel direction vectors and have exactly one common point of intersection.
Parallel and Distinct: The direction vectors are scalar multiples of each other, but the lines share zero common points.
Coincident (Same Line): The direction vectors are scalar multiples of each other, and all points are shared (every point on one line lies on the other).
Skew: The direction vectors are not parallel (not scalar multiples), and the lines share zero common points. Skew lines lie in different parallel planes and never meet.
Geometric Visualizations of Spatial Lines:

Key Geometric Warnings in Three Dimensions:
In , "not parallel" does not imply that two lines intersect (they may be skew).
In , "no intersection" does not imply that two lines are parallel (they may be skew).
Skew lines cannot exist in ; two distinct lines in the plane are either parallel or intersecting.
A visual drawing is not an algebraic test for intersection: Two lines may appear to cross when projected onto a 2D drawing while actually passing each other at different depths or heights.
Systematic Classification Procedure:
Express the lines using distinct parameters:
Step 1: Compare Direction Vectors ( and ):
Check whether for some non-zero scalar .
In , parallel directions can equivalently be checked by verifying .
If directions are parallel:
Test whether the starting point of line lies on by solving .
If a scalar exists: The lines are coincident.
If no scalar exists: The lines are parallel and distinct.
If directions are non-parallel:
Look for a common point by equating coordinates:
This produces a linear system of 3 equations in 2 variables ( and ):
Solve a convenient pair of equations for and , then substitute into the third remaining equation:
If the third equation is satisfied: The system has a unique solution pair , and the lines are intersecting.
If the third equation produces a contradiction: The lines are skew.
Essential Caution: Never set . A shared intersection point may be reached at entirely different parameter values along each line.
Classification Case Studies:
Case 1: Intersecting Lines (Example 1):
Directions: and are not scalar multiples ().
Coordinate Equations:
Since , .
Check third equation: ; . Both sides equal , satisfying the equation.
Find intersection point: Substitute into :
Verification: Substitute into : .
Conclusion: Intersecting at point .
Case 2: Parallel and Distinct Lines (Example 2):
Directions: . Directions are parallel.
Check point on :
Inconsistent scalar (). Point does not lie on .
Conclusion: Parallel and distinct.
Case 3: Coincident Lines (Example 3):
Directions: . Directions are parallel.
Check point on :
Since satisfies all equations, lies on .
Rewriting: .
Conclusion: Coincident (infinitely many shared points).
Case 4: Skew Lines (Example 4):
Directions: Non-parallel.
Coordinate Equations:
Check third equation: Left-hand side is . Right-hand side is . Since , the system has no solution.
Matching and coordinates is insufficient; all three coordinates must match simultaneously.
Conclusion: Skew.
Quick Classification Reference Checks (Against line ):
(a) : Directions and are non-parallel. Equating coordinates yields . Intersecting at point .
(b) : Directions are non-parallel. Equating -coordinates gives (impossible). Skew.
(c) : Direction (parallel). Point has , so it does not lie on . Parallel and distinct.
(d) : Direction (parallel). Point lies on at . Coincident.
Orthogonal Projections and Distance Calculations
Vector Orthogonal Projection Formula:
For a non-zero vector , the orthogonal projection of vector onto is:
Decomposition:
Parallel component: is directed parallel to .
Perpendicular component: The remainder is strictly perpendicular to .
Orthogonality check: .
Distance and Closest Point from a Point to a Line:

Let line pass through base point with direction , and let be an external point.
Total displacement vector: .
Displacement decomposition:
Parallel displacement along the line:
Perpendicular displacement to the line:
Closest Point (): Obtained by starting at point and traversing the parallel displacement:
Shortest Distance: The Euclidean norm of the perpendicular displacement vector:
Category Distinction: The projection is a vector; the closest location is a spatial point; the distance is a non-negative scalar.
Worked Line Projection Problem (Example 19):
Find the distance from point to the line .
Step 1: Displacement vector :
Step 2: Projection of onto :
Step 3: Closest point :
Step 4: Perpendicular remainder and distance:
Distance and Closest Point from a Point to a Plane:

Let be any chosen point lying on plane , let be a vector normal (perpendicular) to , and let be an external point.
Projection onto Normal: Projecting the displacement vector directly onto yields the perpendicular displacement vector connecting to :
Shortest Distance:
Closest Point ():
Procedural Contrast: For a line, project onto the direction vector and subtract to get the perpendicular component. For a plane, projecting onto the normal vector immediately yields the perpendicular component directly.
Worked Plane Projection Problem (Example 20):
Find the distance from point to plane , and determine the closest point .
Identify normal vector from coefficients: .
Select convenient point : Choose , giving .
Displacement vector :
Projection onto normal :
Shortest Distance:
Closest Point :
Verification: Substitute into the plane equation: The point strictly satisfies the plane equation.
Vector Spans and Spanning Sets
Linear Combinations of Vectors:
Standard basis vectors in :
Any vector in decomposes as a unique combination:
Key Connection: Asking whether vector is a linear combination of vectors is equivalent to determining whether the vector equation: possesses a consistent solution for the unknown scalars .
Consistency Test for Linear Combinations:
Construct matrix using the given vectors as columns:
The vector equation is equivalent to the matrix-vector system .
Theorem: The system is consistent if and only if can be written as a linear combination of the columns of .
Worked Linear Combination Problem (Example 7):
Determine whether is a linear combination of:
Set up augmented matrix and row reduce:
Perform :
Perform :
Back substitution:
Row 3: .
Row 2: .
Row 1: .
The system is consistent. Solution: .
Testing Multiple Vectors (Example 8):
Let .
(a) Testing : Consistent. , . Thus, .
(b) Testing : Inconsistent (). Vector cannot be written as a linear combination of and .
Definition of Span (Definition 8):
Let .
The set of all linear combinations of is called the span of :
If , then is designated a spanning set for .
Checkpoint: Span answers the core question: Which vectors can be reached using linear combinations of the given vectors?
Proving a Set Spans (Example 9):
Show that .
Let be arbitrary. Solve :
Because there is a pivot in every row of the coefficient matrix (), the system is consistent for any choices of real numbers and .
Specifically, and .
Therefore, the two vectors span the entirety of .
Geometric Description of Span (Example 10):
Describe .
The two vectors are not scalar multiples of each other, meaning they define two distinct directions.
The span is not all of because two vectors can span at most two dimensions.
Geometrically, the span is a two-dimensional flat plane passing through the origin in .
Inclusion of the Zero Vector:
The zero vector is always present in the span of any non-empty set of vectors.
Reason: Simply assign every scalar weight to zero: .
Connection: This parallels the property that homogeneous linear systems are always consistent because the trivial solution is always valid.
Linear Independence and Rank Criteria
Definition of Linear Independence (Definition 9):
A set of vectors is linearly independent if the vector equation: forces all scalar coefficients to be zero:
If there exists a solution where at least one coefficient is non-zero, the set is linearly dependent.
In a linearly dependent set, at least one vector can be expressed as a linear combination of the other vectors.
Zero Vector Rule: Any set containing the zero vector is automatically linearly dependent.
Justification: If , then provides a non-trivial solution (coefficient ).
Geometric and Structural Significance:
If a vector is already within the span of a set of vectors (as in Example 8(a)), appending it to the set adds no new directional coverage; the span remains unchanged and the set becomes dependent.
If a vector is not in the span (as in Example 8(b)), appending it adds an entirely new dimension/direction to the span, and linear independence is preserved.
Testing Methods for Linear Independence:
Homogeneous-System Test:
Arrange vectors as the columns of matrix .
The vectors are linearly independent if and only if the homogeneous system has exclusively the trivial solution .
If free variables exist (non-trivial solutions exist), the vectors are linearly dependent.
Row-Vector Rank Test:
Arrange vectors in as the rows of a matrix:
The row vectors are linearly independent if and only if (meaning no all-zero rows appear after reduction to REF).
By tracking row operations, a specific dependence relation among dependent rows can be identified directly from the combination producing an all-zero row.
Quick Dimension Test:
Any set of vectors in is guaranteed to be linearly dependent if .
Justification: Placing the vectors as columns of an matrix results in fewer rows than columns (). The rank cannot exceed . Thus, the number of pivots is at most , which guarantees at least free variables in the homogeneous system .
Worked Linear Independence Examples:
Example 11(a):
Solve .
Augmented matrix:
Two pivots, zero free variables. Only works. The set is linearly independent.
Example 11(b):
Augmented matrix:
Three pivots, rank equals . No free variables. Only the trivial solution exists. The set is linearly independent.
Example 11(c) & Example 13 (Row-Vector Rank Test):
Testing vectors as rows in matrix :
Row reduce: and :
Perform :
Here, (a row of zeros appears). The rows are linearly dependent.
Relation tracking: Notice .
Linear relation: .
Example 12 (Standard Basis Independence):
Testing standard basis vectors using columns of :
The homogeneous system is .
Since , this immediately forces .
The standard basis vectors are always linearly independent.
Example 14 (Applying the Quick Dimension Test):
Set: .
The set contains vectors residing in with .
Since , by the Quick Dimension Test, the set is guaranteed to be linearly dependent without needing computation.
Row reduction verification: Two pivots with 3 columns leaves free variable, confirming non-trivial solutions and linear dependence.
Questions & Discussion
Prompt: Can one vector by itself generate every vector in by scalar multiplication?
Response: No. The scalar multiples of a single non-zero vector form the set , which geometrically traces only a one-dimensional line through the origin. It cannot cover the entire two-dimensional area of .
Prompt: Do any two vectors in always generate every vector in ?
Response: No. If the two vectors are collinear (scalar multiples of one another) or if one/both are the zero vector, their linear combinations only span a line or the origin, rather than all of . To generate all of , the two vectors must be non-parallel (linearly independent).
Prompt: Which familiar vectors are guaranteed to span ?
Response: The standard basis vectors (where has a in the -th entry and elsewhere) are guaranteed to span , since any arbitrary vector can be expressed as .
Prompt: Why is always in the span of any non-empty set of vectors? What idea from homogeneous systems does this recall?
Response: The zero vector is always present because setting all scalar weights to zero in a linear combination yields . This mirrors the property of homogeneous systems , which are always consistent because they unconditionally admit the trivial solution .
Prompt: What does the result of Example 8(a) tell us about the three vectors together? Does the third vector add a new direction to the span?
Response: Since is already a linear combination of and , it lies completely within their flat two-dimensional spanning plane. Adding this third vector to the set introduces redundancy, provides no new geometric direction, and renders the three vectors linearly dependent.
Prompt: What happens when the vector from Example 8(b) is added to the set?
Response: Because cannot be written as a linear combination of and , it points outside their two-dimensional plane. Appending it adds a third independent direction, expanding the span from a 2D plane to all of , and keeping the three vectors linearly independent.
Prompt: We have used augmented matrices to organize linear systems. What is a matrix, and where have we seen one before?
Response: A matrix is a rectangular mathematical array of numbers organized into rows and columns. Beyond augmented matrices, matrices appear as coefficient matrices of linear systems, coordinate transformations, and data grids.
Prompt: What should we check before multiplying partitioned matrices?
Response: We must verify conformity between blocks: the column partition sizes of the first matrix must match the row partition sizes of the second matrix so that every required submatrix product and addition is dimensionally defined.
Prompt: For real numbers, implies or . Does force or for matrices? Must equal ?
Response: No to both questions. Two non-zero matrices can multiply to yield the zero matrix , meaning matrix arithmetic contains non-zero divisors of zero. Furthermore, matrix multiplication is non-commutative: does not generally equal , and one product may exist even if the other is undefined.
Prompt: If we have , can we simply "divide by "? What extra concept is needed to solve such an equation?
Response: Division by a matrix is undefined. To isolate , we need the concept of a multiplicative matrix inverse, denoted . If is square and invertible, we multiply both sides on the left by : . If is not square or has no inverse, this technique cannot be applied.
Prompt: What does the word linear mean? What features should a linear equation have?
Response: In algebra, linear signifies that every variable appears only to the first power, is not multiplied or divided by another variable, and does not appear inside transcendental, exponential, or root functions. Geometrically, the solution sets of linear equations form flat geometric objects (lines, planes, or hyperplanes) without curves, kinks, or self-intersections.
Prompt: Why can a linear system not have exactly two solutions?
Response: Geometrically, solutions represent intersections of flat planes or lines. If two flat geometric surfaces intersect at two distinct points, they must intersect along the entire straight line connecting those two points, yielding infinitely many solutions. Algebraically, if and are distinct solutions to , then any convex combination for satisfies: This generates infinitely many distinct solutions, rendering exactly two solutions mathematically impossible.
Prompt: What solution is every homogeneous system guaranteed to have? Can a homogeneous system have no solutions?
Response: Every homogeneous system is guaranteed to have the trivial zero solution . Because the zero vector is always a valid solution, a homogeneous system can never be inconsistent (it can never have zero solutions).
Prompt: If a homogeneous system has equations and variables with , what can be said about the number of solutions?
Response: It is guaranteed to have infinitely many solutions. Because the rank is bounded by the number of equations (), the system must have at least free variables, producing infinitely many non-trivial solutions alongside the trivial solution.