Linear Algebra Study Notes

Goal of the Study

• Understanding of linear transformations, their matrix representations, determinants, eigenvalues/eigenvectors, diagonalization, matrix exponentials, orthogonal projections, least squares, and applications.

Prerequisites

• Familiarity with basic vector space concepts:

  • Vector addition

  • Scalar multiplication

  • Basis

  • Dimension

  • Span

  • Linear independence

• Matrix operations:

  • Addition

  • Multiplication

  • Transpose

  • Inverse

1. Linear Transformations: The Core Concept

• Definition: Linear transformations are functions between vector spaces that preserve structure, meaning:
  $T(av + bw) = aT(v) + bT(w)$
  for any vectors $v, w$ in vector space $V$ and scalars $a, b$ in $R$.

• Key Property: Maps zero vector of input to zero vector of output: $T(0<em>V) = 0</em>W$. Non-linear example: $g(x) = 2x - 2$, since $g(0) ≠ 0$.</p></li></ul><h4 id="0c9d2225-341d-4514-9b55-461fa96ba6c5" data-toc-id="0c9d2225-341d-4514-9b55-461fa96ba6c5" collapsed="false" seolevelmigrated="true">Examples of Linear Transformations</h4><ol><li><p><strong>Example 1</strong>:$T(x) = 3x$</p><ul><li><p>$T(ax + by) = 3(ax + by) = aT(x) + bT(y)$</p></li></ul></li><li><p><strong>Example 2</strong>: If$A$is an$m imes n$matrix,</p><ul><li><p>$T(v) = Av $defines a linear transformation from$R^n$ to $R^m$.</p></li></ul></li><li><p><strong>Example 3</strong>: Differentiation map$D: Pn o P{n-1}, D(p(t)) = p'(t)$.</p></li><li><p><strong>Theorem 49</strong>: A linear transformation is determined by its effect on basis vectors of input space.</p></li></ol><h3 id="19131f13-8b49-4fa6-8bf0-dd13f8a16480" data-toc-id="19131f13-8b49-4fa6-8bf0-dd13f8a16480" collapsed="false" seolevelmigrated="true">2. Coordinate Matrices: Representing Linear Transformations</h3><ul><li><p><strong>Standard Basis</strong>: For$T: R^n o R^m$, standard matrix$A$(often written$T_{E,E}$with$E$as basis) is formed from:</p><ul><li><p>Columns are given by$ T(e1), T(e2), …, T(e_n)$</p></li></ul></li><li><p><strong>General Bases</strong>:</p><ul><li><p>For bases B in$V$and C in$W$:</p></li><li><p>Coordinate Matrix$T{C,B}$is an$m imes n$matrix from$[T(b1)]C, …, [T(bn)]_C$.</p></li></ul></li></ul><h3 id="8cbde864-6572-4863-a2ae-29f3d145ef0e" data-toc-id="8cbde864-6572-4863-a2ae-29f3d145ef0e" collapsed="false" seolevelmigrated="true">3. Determinants: A Special Number Associated with Square Matrices</h3><ul><li><p><strong>Definition</strong>: Determinants provide a scalar value calculated from matrix entries.</p></li><li><p><strong>Key Property</strong>:</p><ul><li><p>$det(A)
  eq 0$indicates that matrix$A$is invertible (Theorem 54).</p></li></ul></li><li><p><strong>Properties</strong>:</p><ul><li><p>$det(I_n) = 1$</p></li><li><p>Row replacement leaves determinant unchanged</p></li><li><p>Row interchange multiplies determinant by -1</p></li><li><p>Row scaling by$s$multiplies determinant by$s$.</p></li></ul></li></ul><h4 id="7ab55edf-66ef-4b18-afd0-9449b8521aeb" data-toc-id="7ab55edf-66ef-4b18-afd0-9449b8521aeb" collapsed="false" seolevelmigrated="true">Calculation</h4><ul><li><p>Use row operations to obtain triangular form. The determinant of a triangular matrix is the product of diagonal entries.</p></li><li><p>Use cofactor expansion for calculations with small matrices.</p></li></ul><h3 id="53ee0b7f-ac66-445b-b939-f856d928a8a4" data-toc-id="53ee0b7f-ac66-445b-b939-f856d928a8a4" collapsed="false" seolevelmigrated="true">4. Eigenvalues and Eigenvectors</h3><ul><li><p><strong>Definition</strong>: An eigenvector of matrix$A$is a non-zero vector$v$such that$Av = 4 v$for some scalar$4$. The vector$v$only changes in magnitude.</p></li></ul><h4 id="ad592083-b0d8-464b-9b84-2925c95159c1" data-toc-id="ad592083-b0d8-464b-9b84-2925c95159c1" collapsed="false" seolevelmigrated="true">Finding Eigenvalues</h4><ul><li><p>Rewrite$Av - 4 I v = 0$leads to non-zero solutions.</p></li><li><p>Condition:$det(A - 4 I) = 0$(Theorem 58).</p></li></ul><h3 id="26aaf89f-bcbf-49c3-9159-508a075ec232" data-toc-id="26aaf89f-bcbf-49c3-9159-508a075ec232" collapsed="false" seolevelmigrated="true">5. Diagonalization</h3><ul><li><p><strong>Definition</strong>: A matrix A is diagonalizable if$A = PDP^{-1}$where D is diagonal (Theorem 66).</p><ul><li><p>Existence of an eigenbasis (collection of eigenvectors) crucial.</p></li><li><p>To diagonalize:</p></li><li><p>Find eigenvalues, determine eigenvectors, construct matrices P and D.</p></li></ul></li></ul><h3 id="f7589d70-8b6d-4667-baeb-19abce4c7107" data-toc-id="f7589d70-8b6d-4667-baeb-19abce4c7107" collapsed="false" seolevelmigrated="true">6. Linear Differential Equations</h3><ul><li><p>Solve$ rac{du}{dt} = Au$with given initial conditions using the eigenvectors of A.</p></li><li><p><strong>Solution</strong>: The unique solution$u(t) = e^{At}v$, is simplified by discoveries of eigenvectors/eigenvalues, yielding exponential growth/decay behavior.</p></li></ul><h3 id="19aa96fb-bd02-4bac-8380-fbaa68773d58" data-toc-id="19aa96fb-bd02-4bac-8380-fbaa68773d58" collapsed="false" seolevelmigrated="true">7. Orthogonality and Projection</h3><ul><li><p><strong>Orthogonal Projection</strong>: Projects vectors onto subspaces preserving linear structure.</p></li><li><p>Understand$projw(v)$and the projection matrix$Pw$.</p></li><li><p><strong>Calculation</strong>:$P_w = rac{1}{w ullet w} ww^T$.</p></li></ul><h3 id="9f07b667-d030-42b2-8ab4-bd95b170716d" data-toc-id="9f07b667-d030-42b2-8ab4-bd95b170716d" collapsed="false" seolevelmigrated="true">8. Least Squares Solutions</h3><ul><li><p>The goal is to minimize distance for systems where solutions don’t exist.</p></li><li><p><strong>Normal Equations</strong>:$A^T(b - Ax) = 0$leads to$A^TAx = A^T b$.</p></li><li><p>Solve for least-squares solution when$A$$ has full column rank.

9. Linear Regression

• Fit linear models to data points using least squares, forming appropriate matrices to calculate coefficients.

Markov Matrices

• Inform about the transitions with the properties of columns summing to one. This aids in analysis pertaining to long-term behavior.

Summary

• Linear operations define, analyze, and simplify complex systems through structured approaches,

  • Applications in solving equations, data fitting, response modeling, and understanding dynamic systems. Good luck with the quiz!