Notes on Linear Algebra: Chapters 1–4 (System of Linear Equations, Euclidean Space, Linear Transformation, Vector Space)
Chapter 1: System of Linear Equations
Overview: Solving linear equations via row operations leads to row echelon form (REF) and reduced row echelon form (RREF). The shape of the final system indicates existence and uniqueness of solutions.
Key objects:
Linear system: A x = b where A ∈ R^{m×n}, x ∈ R^n, b ∈ R^m.
Augmented matrix: (A | b) that encodes the whole system.
Pivot: first nonzero entry in a row of REF/RREF; pivot columns indicate dependent variables.
Free variables: non-pivot columns in A.
1.1 Gauss Elimination
Linear equations are relations of the form x + 3y = 5, 2x + 4y = 6, etc. (Some shown examples prove linear form invariance under reparameterisation.)
Example (two variables): Solve
Subtract 2 times the first equation from the second:
Substitute into first:
Geometric interpretation: two lines intersect at a unique point \((x,y)=(-1,2)).
Two- and three-variable systems illustrate various solution types: unique solution (intersection of lines/planes), no solution (inconsistent), or infinitely many solutions (a line or plane of solutions).
Graphical intuition: a system of two equations in two variables corresponds to two lines; their intersection is the solution. If lines are parallel but distinct, no solution; if coincident, infinitely many solutions.
1.1.1 Linear Equation (intro examples)
Linear equations are those that can be written as a sum of terms where each variable appears to the first power and not multiplied together.
Nonlinear examples: quadratic, cubic, algebraic, transcendental equations.
1.1.2 System of Linear Equations in Two Variables
Gauss elimination: eliminate a variable between equations by forming linear combinations (e.g., Eq2 − 2Eq1).
Example solved:
Given x + 3y = 5 and 2x + 4y = 6, unique solution is \((x,y)=(-1,2)).
Using alternative elimination: 4Eq1 − 3Eq2 gives another path to x = -1, y = 2.
Geometric takeaway: the solution is the intersection of two lines (unless lines are parallel or coincident).
Exercises emphasize reading systems from graphs and eliminating variables algebraically.
1.1.3 System of Linear Equations in Three Variables
Example with three equations in 3 variables: elimination yields a row of zeros (0 = 0) after successive reductions, leaving a parametric solution for free variables (e.g., z arbitrary, x = -2 + z, y = 3 − 2z).
Geometry: solution set is a line in R^3 when one free variable remains.
A slightly modified system may yield a unique solution, or inconsistent system (0 = −1).
Visual: a system with a unique solution corresponds to the intersection of three planes (a single point for the lines of intersection).
1.1.4 Homogeneous System of Linear Equations
Homogeneous system: right-hand side is all zeros. Always has the trivial solution x = 0.
Example: x + 4y + 7z + 10w = 0, 2x + 5y + 8z + 11w = 0, 3x + 6y + 9z + 12w = 0.
Eliminations show that nontrivial solutions exist depending on the rank; with appropriate row operations, we can express the solution as a linear combination of free variables times basis vectors of the null space: e.g., x = z + 2w, y = -2z - 3w, z, w arbitrary.
Matrix perspective: the set of all solutions forms x0 + Null(A), where x0 is a particular solution and Null(A) denotes the null space of A.
1.2 Row Operation
Augmented matrix representation: (A | b) captures both coefficients and constants.
Row operations (swap, multiply a row by a nonzero scalar, add a multiple of one row to another) preserve the solution set.
Example mappings:
Eq2 − 2Eq1 and Eq3 − 3Eq1 correspond to Row2 − 2 Row1 and Row3 − 3 Row1 on the augmented matrix.
Row3 − 2 Row2 corresponds to a further step Row3 − 2 Row2.
Row echelon form (REF) vs Reduced Row Echelon Form (RREF):
REF: rows with leading (pivot) ones move to the right as you go down; zeros below pivots.
RREF: additionally zeros above pivots; pivots are 1 and columns containing a pivot have zeros elsewhere.
Definition 1.2.1: Row echelon form is the “simplest shape” obtainable by row operations; RREF is the simplest matrix obtainable by row operations; pivots define pivot rows and pivot columns.
Observations:
The shape of REF is the same (up to row operations) across many elimination paths.
RREF is unique (for a given augmented matrix).
The end shape is often an upside-down staircase when the pivots move to the right as you go down.
Row operation taxonomy:
1) Rowi ↔ Rowj (exchange rows)
2) cRowi (multiply i-th row by a nonzero scalar c)
3) Rowi + c Rowj (add a scalar multiple of one row to another)
1.2.1 Augmented Matrix (example details)
For a system with two equations in three variables (x, y, z), augmented form example:
A = egin{pmatrix}1 & 4 & 7 \ 2 & 5 & 8 \ 3 & 6 & 9\end{pmatrix}, \ oldsymbol{b} = egin{pmatrix}10 \ 11 \ 12
iar \ (A|oldsymbol{b}) = egin{pmatrix}1 & 4 & 7 & | & 10 \ 2 & 5 & 8 & | & 11 \ 3 & 6 & 9 & | & 12
ight).
The text emphasizes the distinction between the coefficient matrix A and the augmented matrix (A|b).
Throughout, row operations preserve the set of solutions of the corresponding system.
1.2.2 Row Operation (procedures)
Row operations correspond to Gaussian elimination steps:
Row2 − 2Row1, Row3 − 3Row1, Row3 − 2Row2 etc.
Example 1.2.2 shows how different sequences of row operations lead to the same REF or equivalent REFs.
The end result reveals the pivot structure and degrees of freedom (free variables).
1.2.3 Row Echelon Form
Refines the idea of REF: pivots occur in a staircase down-right pattern.
The pivots help determine solution structure: number of pivots equals rank of A, etc.
The end goal is to reduce to a form where solution reading is straightforward.
1.3 Existence and Uniqueness
Two central questions for any A x = b:
1) Existence: Does a solution exist for given b?
2) Uniqueness: If a solution exists, is it unique?Theorem 1.3.1 (Existence): A x = b has a solution iff b is not a pivot column of the augmented matrix (A | b).
If b lies in a pivot column, the system is inconsistent (no solution).
Reading the solution from RREF: if all columns of A are pivots, the solution is unique; otherwise free variables remain.
Theorem 1.3.2 (Uniqueness): For a matrix A, the following are equivalent:
1) The solution of A x = b is unique.
2) A x = 0 has only the trivial solution x = 0.
3) All columns of A are pivot columns.Theorem 1.3.3 (Existence for all right sides): A x = b has a solution for all b iff all rows of A are pivots in REF (no zero row in A, i.e., rank(A) = m).
Example 1.3.4 discusses a matrix A where the condition for A x = b to have a solution for all b is rank(A) = m (A has full row rank). If A is square (m = n), then full rank means A is invertible.
The key dichotomy: The right-hand side b only affects the existence. The structure of A (rank) governs existence for all b and uniqueness for a given b.
1.3.2 Criteria for Existence and Uniqueness
Theorem 1.4.2 (Rank formulation): For A ∈ R^{m×n} and b ∈ R^m,
1) A x = b has a solution iff rank([A|b]) = rank(A).
2) A x = b has a solution for all b iff rank(A) = m (i.e., rows of A are pivot).
3) The solution is unique iff rank(A) = n (i.e., all columns pivot).Theorem 1.4.4 (Two of three imply the third): For a square matrix A, if any two of (A is square, A x = b has a solution for all b, the solution is unique) hold, then the third holds as well.
Theorems connect the existence/uniqueness questions to rank and pivot structure.
The rank concept is the “essential size” of a system; it captures the essential equations after redundant ones are eliminated.
1.3.3 Criteria for Existence for All Right Side
Example 1.3.5 and subsequent: Criteria for A x = b to have a solution for all b is equivalent to all rows of A being pivots in REF (no zero row). This ensures the augmented system has no contradictions for any b.
Theorem 1.3.3 and Theorem 1.3.2 unify the idea: A x = b solvable for all b iff rank(A) = m and the columns are sufficient to express any b, etc.
1.3.4 Criteria for Existence for a Given b
Example 1.3.4 shows dependence on a parameter a in the augmented system. Cases:
If a ≠ 9, there is a unique solution for all b; if a = 9, existence may depend on b (e.g., b must satisfy a linear equation to be solvable).
The discussion emphasizes how augmented forms influence the existence of solutions for particular b. The last column in the echelon form indicates contradictions (0 = c ≠ 0) or a valid equation.
1.3.5 Criteria for Existence for All Right Sides (Revisited)
Theorem 1.3.5 (for all right sides) and Theorem 1.3.3 (on rows) cohered into Theorems 1.3.5–1.3.6 in the text; the essence remains: full row rank yields existence for all b; full column rank yields uniqueness for all b.
1.4 Rank
Rank is the pivot count in REF; it equals the dimension of the column space Col(A) and equals the dimension of the row space Row(A).
Essential size: rankA is the size of the core in which nonredundant information lives.
Fundamental relationships:
rank(A) ≤ min{m, n}.
If rank(A) = m, A has full row rank; if rank(A) = n, A has full column rank.
If rank(A) = min{m, n}, A has full rank.
Theorem 1.4.2 (Rank characterization): various equivalences linking the existence/uniqueness of solutions to rank conditions.
Example 1.4.1 analyzes a 3×4 matrix and shows how rank depends on a parameter a, yielding rank 3 or 2 depending on a and b.
Theorem 1.4.3: For A ∈ R^{m×n}, if a system Ax = b has a solution for all b, then m ≤ n. If the solution is unique, then m ≥ n. This aligns with intuitive counting: to uniquely determine n variables we need at least n equations; if you have more equations than variables, you generally cannot satisfy all of them for all b.
Theorem 1.4.4 (Two of three imply the third for a square A): square matrix A is invertible iff it has full rank; equivalently, iff A x = b has a unique solution for all b, or iff there exists B with AB = BA = I.
Chapter 2: Euclidean Space
2.1 Euclidean Vector
Definition: The Euclidean space R^n is the set of all n-tuples of reals; vectors are elements of R^n.
Operations:
Addition:
Scalar multiplication:
The origin is the zero vector \boldsymbol{0} = (0,0,…,0).
Basic identity: The left-hand side of a system, A x, can be seen as a linear combination of the columns of A: if A = [v1 v2 … vn], then A x = x1 v1 + x2 v2 + … + xn vn. The set Col(A) = {A x : x ∈ R^n} is the column space of A.
The standard basis of R^n is ei with ei having a 1 in the i-th position and 0 elsewhere.
Example 2.1.1: The general solution to the two-equation, three-variable system x1+4x2+7x3 = 10, 2x1+5x2+8x3 = 11 is expressed by a particular solution plus a multiple of a vector in the null space, showing a line of solutions in R^3.
Example 2.1.2: General solution to a two-equation system can be written as x = x0 + t v, a line in R^n. For the system in Example 1.1.2, x = (-2,3,0) + z(1,-2,1).
Example 2.1.3–2.1.4 discuss embedding lines, and interpreting the system as a linear combination of columns of A to express the vector b as a combination of v_i.
2.1.2 Left Side A x of System of Linear Equation
The left side of A x = b is a linear combination of the column vectors of A: A x = x1 v1 + … + xn vn.
Proposition 2.1.2: A(a x + b y) = a A x + b A y; linear maps preserve linear combos.
Exercises 2.4–2.7 explore expressing b as a linear combination of columns and non-uniqueness/uniqueness depending on independence of the columns.
2.2 Span and Linear Independence
Span: Span(α) = {∑ xi vi : x_i ∈ R} for a set α = {v1, …, vn}.
Column space Col(A) = Span{v1, …, vn} where A = [v1 … vn].
Linear independence: a set α is linearly independent iff the equation ∑ xi vi = 0 implies all coefficients x_i = 0.
Two vectors are linearly independent iff they are not parallel in R^2; three vectors may be dependent if one is a linear combination of the others (e.g., (1,2,3), (4,5,6), (7,8,9) in R^3 are dependent).
Example 2.2.2 shows how algebraic dependence of columns (via a row-reduction of [A|b]) determines whether Col(A) equals all of R^m or a subspace defined by a constraint (e.g., b3 − 2 b2 + b1 = 0 when a = 12 in a 3×4 matrix).
2.3 Subspace of R^m
Subspace: A subset H ⊂ R^m is a subspace if it contains 0 and is closed under addition and scalar multiplication.
ColA and NulA are subspaces of R^m and R^n, respectively.
Null space NulA = {x ∈ R^n : A x = 0} is the set of all solutions to the homogeneous system; ColA is the subspace spanned by the columns of A.
Example 2.3.1 and 2.3.2 illustrate subspaces such as lines through the origin and the unit disk (the latter is not a subspace) and the need for closure under scalar multiplication.
Exercise 2.17–2.19 test subspace properties on various subsets of R^2.
2.3.2 General Solution is Shift of Null Space
For a system A x = b with a particular solution x0, the full solution set is x = x0 + NulA.
If the augmented matrix has a RREF with nonzero last column entries, the system may have a unique solution; otherwise, the solution forms an affine subspace (a translate of the null space).
Example 2.3.3 demonstrates how the solution space is x0 + Span{v1, v2} when the null space is two-dimensional.
Proposition 2.3.2 and 2.3.3 formalize:
If there exists x0 with A x0 = b, then all solutions are x0 + NulA.
The solution is unique iff NulA = {0}.
2.4 Basis
Basis: A finite set α is a basis for a subspace H if α spans H and is linearly independent.
Theorem 2.4.1 (Equivalent definitions of a basis): α spans H and is independent ⇔ α is a minimal spanning set ⇔ α is a maximal independent set.
A basis is the smallest set of vectors that spans the subspace; the number of vectors in a basis is the dimension of the subspace, dim H.
Example 2.4.1: The standard basis of R^n is a basis, and any vector x ∈ R^n can be written x = ∑ xi ei; [x]{ε} = (x1, …, x_n).
Example 2.4.2–2.4.3 show how to obtain a basis for ColA via pivot columns and discuss the relation to the rank of A.
Proposition 2.4.2: dim ColA = rank A; rank α is defined as dim Span α.
Exercise 2.31–2.36 develop practical basis construction for various A and collect exercises on ColA, RowA, NulA, and NulAT.
2.4.2 Basis of Column Space: First Method
Find a basis of ColA by performing row operations on A and looking at pivot columns. Pivot columns of A form a basis of ColA; nonpivot columns are linear combinations of pivot columns.
Example 2.4.3 demonstrates this approach on a 3×4 matrix, yielding a basis {v1, v2} for ColA in that scenario, after pivot columns 1 and 2 remain.
2.4.3 Property of Dimension
Theorem 2.4.3: If H′ ⊂ H, then dim H′ ≤ dim H; if dim H′ = dim H, then H′ = H. This ensures all bases of a finite-dimensional subspace have the same size.
Theorem 2.4.4: For any subspace, if α spans H, then |α| ≥ dim H; if α is independent, then |α| ≤ dim H; equality implies α is a basis.
Example 2.4.4 demonstrates that (1,2) and (3,4) are independent in R^2 and form a basis, implying that the corresponding 2×2 system has a unique solution for all right-hand sides.
2.4.4 Basis of Null Space
The null space NulA is spanned by the free-variable vectors derived from the RREF of A (e.g., NulA has a basis {v1, v2}).
Example 2.4.5 shows NulA for a 3×4 example is spanned by two independent vectors.
Proposition 2.4.5: dim NulA = n − rank A (n = number of columns in A).
The dimension balance: dim ColA + dim NulA = n.
2.4.5 Basis of Column Space: Second Method
The row space is RowA ⊂ R^n, and ColA is the RowA^T; the pivot columns of At give a basis for RowA, and pivot rows give a basis for ColA under transposition considerations.
Theorem 2.4.7: rank(AT) = rank(A).
The text demonstrates how to obtain a basis for the row space via column operations on A^T and pivot columns of A as well as using row operations on A to obtain a column-echelon form.
2.5 Sum and Direct Sum
Generalisation of span and independence to subspaces: H1 + H2 + … + Hn is the set of all sums x1 + x2 + … + xn with xi ∈ Hi.
Direct sum H1 ⊕ H2 ⊕ … ⊕ Hn means the representation is unique; equivalently, the intersection H_i ∩ (sum of the others) is {0} for all i.
For one-dimensional Hi = R vi, we have Span{vi} as the direct sum iff the vectors are linearly independent.
Theorem 2.5.5 gives a general inequality: dim(H1 + H2 + … + Hn) ≤ ∑ dim Hi, with equality iff the sum is direct and the basis α1 ∪ … ∪ αn forms a basis for the sum.
For two subspaces, dim(H1 + H2) = dim H1 + dim H2 − dim(H1 ∩ H2).
Propositions 2.5.2 and 2.5.3 detail conditions for non-direct sums and nontrivial intersections.
Layers of Direct Sum: equivalence between directness of partial sums and the whole sum.
Chapter 3: Linear Transformation
3.1 Matrix of Linear Transformation
Three viewpoints: equation form A x = b; vector form; matrix form as a linear transformation L: R^n → R^m.
Transformation given by a matrix: L(x) is linear, and A = [L] is constructed by applying L to the standard basis vectors: A = [L(e1) L(e2) … L(en)]. Then L(x) = A x.
Example 3.1.1–3.1.4 illustrate basic transformations (flip, identity, antipode) and how to obtain their matrices.
Example 3.1.5–3.1.7 shows special transformations like projection, derivative, and rotation, and how to express them as matrices in a chosen basis.
Elementary matrices: Row operations on a vector x correspond to multiplying by an elementary matrix E on the left: E x applies the row operation to x.
3.1.1 Transformation Given by Matrix
A transformation is linear iff it respects addition and scalar multiplication; the matrix representation is built by mapping basis vectors to images under the transformation.
The identity transformation has matrix I, and the zero transformation has matrix 0.
3.1.2 Linear Transformation is Equivalent to Matrix
Theorem: There is a one-to-one correspondence between linear transformations L: R^n → R^m and m×n matrices A: L(x) = A x and A = [L].
The columns of A are the images of the standard basis under L.
3.1.3 Calculation of Matrix of Linear Transformation
Given L and vectors v1, v2, …, vn with L(v_i) computed, one can assemble the matrix by placing these images as columns in the output basis.
Practical method: Solve for L(ei) by expressing ei as a linear combination of a set of vectors whose images are known, etc.
3.2 Matrix Operation
Sum of linear transformations corresponds to sum of matrices: (L+K)(x) = L(x) + K(x).
Scalar multiples: (cL)(x) = c L(x).
Matrix form respects these rules: [L+K] = [L] + [K], [cL] = c[L].
Matrices act as a way to encode linear maps; in particular, the rows of a matrix can be viewed as linear functionals applied to x.
Exercise 3.14 and 3.15 explore explicit additions of matrices and linear transformations.
3.2.2 Multiplication of Matrix is Composition of Linear Transformations
Composition L ◦ K corresponds to matrix multiplication: [L ∘ K] = [L] [K], with the caveat that dimensions must align (the output dimension of K matches the input dimension of L).
Example with 2×2 matrices shows how multiplication corresponds to composing the linear maps.
The associativity of composition corresponds to the associativity of matrix multiplication: (AB)C = A(BC).
Note: Composition of transformations is generally not commutative, mirroring AB ≠ BA in general.
The section also remarks how left/right multiplication by matrices corresponds to applying L to columns or rows of a matrix.
3.3 Onto and One-to-One
Image (Ran L) and Preimage (L^{-1}(·)) interpretations for linear transformations.
A linear map L is onto iff Ran L = R^m; equivalent to A x = b being solvable for every b.
A linear map is one-to-one iff Ker L = {0}; equivalent to A x = b having at most one solution for any b (and unique solution when it exists for all b).
Propositions 3.3.1 and 3.3.2 provide equivalences tying onto/one-to-one to column/row/pivot properties of the matrix A and the vectors spanning the respective subspaces.
3.3.1 Image and Preimage of Map
Ran L corresponds to the column space of A. Kernel corresponds to solutions to A x = 0.
Useful dictionary: the properties of L translate to properties of A (injective/surjective) via rank and nullity considerations.
3.4 Inverse
Inverse concept: A map L is invertible if there exists K with K ∘ L = I and L ∘ K = I.
Theorem 3.4.1: A map is invertible iff it is onto and one-to-one.
For matrices, A is invertible iff there exists B with AB = BA = I; for square A, invertibility is equivalent to having full rank.
Example 3.4.2 shows 2×2 and 3×3 cases; the inverse is computed via block/row operations (A I) → (I B).
The chapter emphasizes the dictionary between linear algebra concepts and the matrix world; determinants are not introduced here but are implicit in invertibility and pivot structure.
3.4.2 Inverse Matrix
The inverse of a square matrix A, if it exists, is A^{-1} with AB = BA = I.
The example shows explicit computation and the link to row operations to reach I on the left and B on the right.
3.5 Block Matrix
Block matrices generalize simple 2×2 blocks to larger block decompositions; many results (sum/product/inverses) carry over with appropriate block sizes.
The text provides properties and examples of block matrices and their inverses in terms of sub-blocks A, B, C, D.
3.6 LU-Decomposition
LU-decomposition expresses a matrix A as A = LU, with L lower triangular and U upper triangular.
Gaussian elimination can be viewed as multiplying A on the left by a product of lower triangular elementary matrices: Ek … E2 E1 A = U, so A = L U with L = E1^{-1} E2^{-1} … Ek^{-1}.
If row operations require row exchanges (to avoid zeros on pivots), a permutation P is introduced: P A = LU.
The section shows how to identify L from the sequence of row operations and explains how the numbers used in Row operations appear in L.
Exercises 3.36–3.37 explore finding LU decompositions with and without permutations.
Chapter 4: Vector Space
4.1 Vector Space
Goal: Extend linear algebra beyond Euclidean geometry to general vector spaces, preserving linear combination operations.
Axioms (Definition 4.1.1): A real vector space V has binary operation of addition and scalar multiplication satisfying eight axioms:
Commutativity: x + y = y + x
Associativity of addition: (x + y) + z = x + (y + z)
Zero vector: There exists 0 ∈ V with x + 0 = x = 0 + x
Existence of additive inverses: For each x ∈ V there exists −x with x + (−x) = 0
Identity of scalar multiplication: 1 x = x
Associativity of scalar multiplication: (ab)x = a(bx)
Distributivity over scalar addition: (a + b)x = a x + b x
Distributivity over vector addition: a(x + y) = a x + a y
Examples (4.1.2–4.1.4): finite-dimensional polynomial space P_n, matrix space M(m,n), sequence space, function spaces (Fun(X), C∞, C[0,1]), etc.
Subtlety: some proposed sets do not form subspaces (e.g., a subspace must contain the zero vector).
4.1.2 Subspace
Definition 4.1.2: A subset H ⊂ V is a subspace if it is closed under linear combinations: for any x, y ∈ H and scalars a, b ∈ R, a x + b y ∈ H.
Subspaces inherit the vector space operations from V.
In Euclidean space, ColA, RowA, NulA are subspaces and often of interest.
4.2 Basis
Ordered basis: An ordered set α = {v1, v2, …, vn} is a basis if every element of V can be written uniquely as a linear combination x = ∑ xi vi (i = 1..n).
The coefficient vector [x]_α ∈ R^n records the coordinates of x with respect to α.
Theorem 4.2.3 (equivalence of bases): A set α is a basis iff it is a minimal spanning set iff it is a maximal linearly independent set.
Existence of bases in finite-dimensional spaces: every finite-dimensional subspace has a basis; infinite-dimensional spaces may not have finite bases.
Coordinate maps: Given a basis α, there is a coordinate isomorphism [·]_α : V → R^n.
4.2.1 Ordered Basis
Emphasizes how changing the order of a basis changes the coordinate representation; the coordinate map depends on the order.
4.2.2 Coordinate with Respect to Ordered Basis
The map [x]α is the coordinate vector; its properties preserve linear combinations: [a x + b y]α = a [x]α + b [y]α.
The matrix of a linear transformation with respect to bases α (domain) and β (codomain) is defined as [L]_{βα} which encodes the action of L on α expressed in β.
If α is a basis of V and β is a basis of W, then the matrix [L]{βα} is constructed by expressing L(vi) in the β-basis and placing those coordinate vectors as columns.
4.2.3 Use Coordinate to Solve Linear Algebra Problem
Translating problems between vector spaces and Euclidean spaces via [·]_α allows the use of matrix methods to solve abstract vector-space problems.
Example: The evaluation map L: P2 → R^3 given by L(f) = (f(t0), f(t1), f(t2)) is onto when the Vandermonde matrix associated with t0, t1, t2 is invertible, and the inverse L^{-1} constructs the interpolating polynomials.
4.3 Linear Transformation
Linear transformation between general vector spaces: L: V → W is linear if L(x + y) = L(x) + L(y) and L(a x) = a L(x).
The set Hom(V, W) of all linear transformations is itself a vector space under pointwise addition and scalar multiplication.
4.3.1 Linear Transformation Between General Vector Spaces
Examples: identity I, zero map; evaluation maps; derivative maps in function spaces; transpose as a linear map on matrices.
The matrix of a linear transformation relative to bases α (domain) and β (codomain) is defined and denoted [L]_{βα}.
The concept of the algebra of linear maps and the isomorphisms between Hom(V,W) and M(m,n) when V ≅ R^n and W ≅ R^m.
4.3.2 Operations of Linear Transformations
Addition, scalar multiplication, and composition of linear transformations behave as expected; their matrices obey corresponding algebra, i.e., [L + K] = [L] + [K], [L ∘ K] = [L][K], etc.
Examples exhibit the translation from transformations to matrix equations.
3.3 Onto and One-to-One (Revisited; in broader context)
Onto/Injective correspond to column space covering codomain and trivial kernel respectively.
Inverse transformations exist iff the map is both onto and one-to-one; this corresponds to a square matrix being invertible.
3.4 Inverse
See above: invertible if and only if there exists a two-sided inverse; equivalence holds in the finite-dimensional/finite-rank case.
Inverse is handled via A^{-1} when appropriate; properties such as A B = I imply solvability for all b as B b is a solution.
3.5 Block Matrix
Block matrix syntax and rules generalize to multiple subspaces; block multiplication aligns with composition of linear maps acting on subspaces.
3.6 LU-Decomposition
A = LU with L lower triangular and U upper triangular; used to perform Gaussian elimination in a structured way.
When row swaps are needed, a permutation matrix P is used and the factorization becomes P A = L U.
The LU decomposition provides a convenient framework for solving Ax = b and understanding elimination as matrix multiplication by L and U.
Chapter 4: Vector Space (continued)
4.1.3 Translate Linear Algebra Problem to Euclidean Space
The chapter explains how to translate questions in abstract vector spaces to Euclidean spaces using an ordered basis and the coordinate map to leverage the easier linear-algebraic tools in R^n.
Examples include checking whether a given set of polynomials spans P2 via linear systems, and translating a question about linear independence to a question about pivots in a matrix.
4.2 Basis (continued)
Infinite and finite dimensional vector spaces: The dimension of a finite dimensional subspace is the size of any of its bases, and the dimension is well defined (Theorem 4.2.7/4.2.8).
Examples show the computation of basis and dimension for spaces of polynomials, matrices, and other vector spaces.
4.3.4 Isomorphism
Two vector spaces are isomorphic if there exists a bijective linear map between them.
If dim V = dim W, there exist coordinate isomorphisms [·]{α} and [·]{β} which identify V and W with R^n; the composition yields an isomorphism between V and W.
Analysis: isomorphism preserves all linear-algebraic structure; many results transfer between isomorphic spaces.
4.3.6 Change of Basis
The matrix of a linear transformation with respect to different bases changes by conjugation with change-of-basis matrices: [L]{ββ} = [I]{βα} [L]{αα} [I]{αβ}.
The change-of-basis matrix [I]_{αα′} is the matrix of expressing the α′-basis in terms of α (or vice versa).
Examples illustrate the computation of these matrices and how a simple problem in one basis becomes easier in another.
4.3.7 Trace and Other Invariants
The trace tr(A) is linear and invariant under similarity: tr([L]{ββ}) = tr([L]{αα}) if α and β are bases related by an invertible change of basis. This makes trace an intrinsic property of the linear operator, not dependent on basis choice.
The text also discusses properties like symmetry, projections, and other subspace decompositions as linear operators and their matrix representations.
4.3.8 Examples and Exercises
Numerous worked examples (including Lagrange interpolation, projection matrices, and basis-change matrices) illustrate the abstract machinery with concrete computations.
Exercises (e.g., Exercise 4.33 to 4.53) probe the construction of bases, coordinates, change-of-basis matrices, and matrix representations of linear maps in different bases.
Key Formulas and Concepts (summary with LaTeX)
System and augmented matrix:
Row operations preserve solution sets; REF and RREF define pivot structure.
Existence/Uniqueness (rank-based):
Existence: a system Ax = b has a solution iff rank([A|b]) = rank(A).
All b solvable iff rank(A) = m (rows pivot).
Unique solution iff rank(A) = n (columns pivot).
Null space and solution sets:
If x0 is a particular solution of Ax = b, then all solutions are
If Nul(A) = {0}, the solution is unique; otherwise, free parameters describe a family of solutions.
Rank and dimensions:
Vector-space axioms (Def. 4.1.1): eight axioms (closure, associativity, identity, inverses, distributivity, etc.).
Basis and dimension: A finite-dimensional vector space V has a basis α with |α| = dim V; coords [x]_α give the coordinates of x in V; isomorphisms to R^n exist when dim V = n.
Linear transformation and matrix: L: V → W is linear iff L(x + y) = L(x) + L(y) and L(a x) = a L(x). If α and β are bases for V and W, the matrix [L]{βα} encodes L in those bases: [L]{βα} [x]{α} = [L(x)]{β}.
Change of basis: If α′ and β′ are other bases, [L]{β′α′} = [I]{βα} [L]{βα} [I]{α′α} etc; base-change matrices relate coordinates across bases.
Direct sum: H1 ⊕ H2 ⊕ … ⊕ Hn is direct iff the only representation of 0 as a sum of hi ∈ Hi is hi = 0 for all i; equivalently, the union of bases αi for each Hi is a basis for the sum.
LU-decomposition: A = LU (possibly PA = LU with a permutation P); L is lower triangular, U is upper triangular; link to Gaussian elimination.
Inverse and determinant-free viewpoint: A invertible iff ∃ B with AB = BA = I; for square A, A invertible ⇔ det(A) ≠ 0 (not stated explicitly but used in matrix theory).
Notes for exam preparation:
Be able to perform Gaussian elimination to obtain REF and RREF, and read off existence/uniqueness from the RREF.
Use rank to decide about solvability for all right-hand sides vs. solvability for a particular right-hand side.
Distinguish ColA, NulA, RowA, and RowA^T concepts; know how to compute bases for ColA and NulA via pivot columns/variables.
Understand the translation between Euclidean coordinates and abstract vector spaces via a basis; be able to compute coordinate representations and invert bases when needed.
Master the matrix interpretation of linear transformations, including composition, inverse, and change of basis techniques.
Practice LU-decomposition by row-operations perspective and by permutation; know when P A = LU is required.
Grasp the concept of direct sums and the dimension formula for the sum of subspaces: dim(H1 + H2) = dim H1 + dim H2 − dim(H1 ∩ H2).
Be comfortable with the concept of rank and its relationship to the existence/uniqueness of solutions, as well as the invariance of rank under transposition: rank(A) = rank(A^T).
If you’d like, I can turn this into a compact study sheet with the most essential definitions, theorems, and a few representative worked examples for quick revision.