Linear Algebra Study Notes
Goal of the Study
Understanding of linear transformations, their matrix representations, determinants, eigenvalues/eigenvectors, diagonalization, matrix exponentials, orthogonal projections, least squares, and applications.
Prerequisites
1. Linear Transformations: The Core Concept
Definition: Linear transformations are functions between vector spaces that preserve structure, meaning:
T(av+bw)=aT(v)+bT(w)
for any vectors $v, w$ in vector space $V$ and scalars $a, b$ in $R$.
Key Property: Maps zero vector of input to zero vector of output: T(0<em>V)=0</em>W. Non-linear example: $g(x) = 2x - 2$, since $g(0) ≠ 0.</p></li></ul><h4id="0c9d2225−341d−4514−9b55−461fa96ba6c5"data−toc−id="0c9d2225−341d−4514−9b55−461fa96ba6c5"collapsed="false"seolevelmigrated="true">ExamplesofLinearTransformations</h4><ol><li><p><strong>Example1</strong>:T(x) = 3x</p><ul><li><p>T(ax + by) = 3(ax + by) = aT(x) + bT(y)
Example 2: If $A$ is an $m imes n$ matrix,
T(v) = Av definesalineartransformationfromR^n$ to $R^m.</p></li></ul></li><li><p><strong>Example3</strong>:DifferentiationmapD: Pn o P{n-1}, D(p(t)) = p'(t).</p></li><li><p><strong>Theorem49</strong>:Alineartransformationisdeterminedbyitseffectonbasisvectorsofinputspace.</p></li></ol><h3id="19131f13−8b49−4fa6−8bf0−dd13f8a16480"data−toc−id="19131f13−8b49−4fa6−8bf0−dd13f8a16480"collapsed="false"seolevelmigrated="true">2.CoordinateMatrices:RepresentingLinearTransformations</h3><ul><li><p><strong>StandardBasis</strong>:ForT: R^n o R^m, standard matrix $A$ (often written $T_{E,E}$ with $E$ as basis) is formed from:
General Bases:
For bases B in $V$ and C in $W$:
Coordinate Matrix T{C,B} is an $m imes n$ matrix from [T(b1)]C, …, [T(bn)]_C.</p></li></ul></li></ul><h3id="8cbde864−6572−4863−a2ae−29f3d145ef0e"data−toc−id="8cbde864−6572−4863−a2ae−29f3d145ef0e"collapsed="false"seolevelmigrated="true">3.Determinants:ASpecialNumberAssociatedwithSquareMatrices</h3><ul><li><p><strong>Definition</strong>:Determinantsprovideascalarvaluecalculatedfrommatrixentries.</p></li><li><p><strong>KeyProperty</strong>:</p><ul><li><p>det(A)
eq 0 indicates that matrix $A$ is invertible (Theorem 54).
Properties:
Calculation
4. Eigenvalues and Eigenvectors
Finding Eigenvalues
Rewrite Av - 4 I v = 0leadstonon−zerosolutions.</p></li><li><p>Condition:det(A - 4 I) = 0(Theorem58).</p></li></ul><h3id="26aaf89f−bcbf−49c3−9159−508a075ec232"data−toc−id="26aaf89f−bcbf−49c3−9159−508a075ec232"collapsed="false"seolevelmigrated="true">5.Diagonalization</h3><ul><li><p><strong>Definition</strong>:AmatrixAisdiagonalizableifA = PDP^{-1}whereDisdiagonal(Theorem66).</p><ul><li><p>Existenceofaneigenbasis(collectionofeigenvectors)crucial.</p></li><li><p>Todiagonalize:</p></li><li><p>Findeigenvalues,determineeigenvectors,constructmatricesPandD.</p></li></ul></li></ul><h3id="f7589d70−8b6d−4667−baeb−19abce4c7107"data−toc−id="f7589d70−8b6d−4667−baeb−19abce4c7107"collapsed="false"seolevelmigrated="true">6.LinearDifferentialEquations</h3><ul><li><p>Solve rac{du}{dt} = AuwithgiveninitialconditionsusingtheeigenvectorsofA.</p></li><li><p><strong>Solution</strong>:Theuniquesolutionu(t) = e^{At}v, is simplified by discoveries of eigenvectors/eigenvalues, yielding exponential growth/decay behavior.
7. Orthogonality and Projection
Orthogonal Projection: Projects vectors onto subspaces preserving linear structure.
Understand $projw(v)$ and the projection matrix $Pw$.
Calculation: P_w = rac{1}{w ullet w} ww^T.</p></li></ul><h3id="9f07b667−d030−42b2−8ab4−bd95b170716d"data−toc−id="9f07b667−d030−42b2−8ab4−bd95b170716d"collapsed="false"seolevelmigrated="true">8.LeastSquaresSolutions</h3><ul><li><p>Thegoalistominimizedistanceforsystemswheresolutionsdon’texist.</p></li><li><p><strong>NormalEquations</strong>:A^T(b - Ax) = 0leadstoA^TAx = A^T b.</p></li><li><p>Solveforleast−squaressolutionwhenA$$ has full column rank.
9. Linear Regression
Markov Matrices
Summary