Comprehensive Study Notes: Linear Algebra - Matrix Operations, Vectors in R^n, and Dot Products

Class Logistics and Resources

• Recording Locations: All class recordings are stored in the same location as the class Zoom meeting links. Above the list of meetings, there is a tab for "cloud recordings"; this is where the archive of recorded lectures is housed.

• Lecture Notes: Written lecture notes are posted directly on the front page of the Canvas site. Students should look there for written material while visiting the Zoom page specifically for video content.

• Attendance Policy: Frequent attendance is required. Missing three meetings may result in being dropped from the course. Students must participate with cameras on during exams, and exams must be completed using paper and pencil on a stable internet connection.

Review of Section 2.1: Matrix Operations

• Previously Covered Content: The course has already introduced fundamental operations: addition, subtraction, scalar multiplication, and the initial concept of matrix multiplication.

• Matrix Multiplication Intuition: Matrix multiplication is derived from expressing a system of linear equations in matrix form.

• Example Case Study: Consider a two-by-two system:   - $x + 2y = 3$   - $4x + 5y = 6$

• Matrix Equation Representation: This system can be rewritten as a matrix equation of the form $AX = B$, where:   - The coefficient matrix $A$ is $\begin{pmatrix} 1 & 2 \ 4 & 5 \end{pmatrix}$.   - The variable matrix (column matrix) $X$ is $\begin{pmatrix} x \ y \end{pmatrix}$.   - The constant matrix (column matrix) is $\begin{pmatrix} 3 \ 6 \end{pmatrix}$.

• Row-Times-Column Multiplier: To reconstruct the original equations from the matrix form, one must multiply the row of the first matrix by the column of the second. This produces the expression $1 \times x + 2 \times y$.

• The Role of Addition: In matrix multiplication, the values from the row and column are added together specifically to replicate the structure of linear algebraic expressions.

Dimension Requirements for Matrix Multiplication

• Inner and Outer Dimensions: For matrix multiplication to be defined, the number of columns in the first matrix must match the number of rows in the second.   - If matrix $A$ is size $m \times k$ and matrix $B$ is size $k \times n$, the inner numbers ($k$) match.   - The resulting matrix dimension is determined by the outer numbers: $m \times n$.

• Commutativity Warning: Matrix multiplication is not always commutative. Generally, $AB \neq BA$.   - Specific Exceptions: Commutativity may occur in rare instances, such as if one matrix is the zero matrix ($O$). Multiplying by a zero matrix on either the left or right will consistently yield a zero matrix, but this is an exception rather than the rule.

Section 2.1 Continued: The Dot Product (Scalar Product)

• Definition: The dot product is the formal name for the "row dotted with a column" operation. It takes two vectors of the same size and outputs a single scalar value.

• Mechanism: Each component is multiplied by its corresponding partner, and the products are summed.

• Numerical Example: Calculate the dot product of two vectors:   - Vector 1 (Row): $(2, -1, 3, -4)$   - Vector 2 (Column): $(5, 2, -3, 1)$   - Calculation: $(2 \times 5) + (-1 \times 2) + (3 \times -3) + (-4 \times 1)$   - Step-by-step: $10 - 2 - 9 - 4$   - Result: $-5$

General Definition of Matrix Multiplication

• A-Matrix Subscripts: Represent matrix $A$ (size $m \times k$) as a set of row vectors $A = \begin{pmatrix} a_1 \ a_2 \ \vdots \ a_m \end{pmatrix}$, where each $a_i$ is a $1 \times k$ row vector.

• B-Matrix Subscripts: Represent matrix $B$ (size $k \times n$) as a set of column vectors $B = \begin{pmatrix} b_1 & b_2 & \dots & b_n \end{pmatrix}$, where each $b_j$ is a $k \times 1$ column vector.

• Resulting Matrix $C$: The entry $c_{ij}$ (the result at the $i^{th}$ row and $j^{th}$ column) is found by: $c_{ij} = a_i \cdot b_j$.

• Expanded Visualization: The product $AB$ results in an $m \times n$ matrix of the form:   - $\begin{pmatrix} a_1 \cdot b_1 & a_1 \cdot b_2 & \dots & a_1 \cdot b_n \ a_2 \cdot b_1 & a_2 \cdot b_2 & \dots & a_2 \cdot b_n \ \vdots & \vdots & \ddots & \vdots \ a_m \cdot b_1 & a_m \cdot b_2 & \dots & a_m \cdot b_n \end{pmatrix}$

Linear Combinations

• Vocabulary Definition: A linear combination is the sum of scalar multiples of elements within a specific set.

• Mathematical Expression: Given a set of objects ${x_1, x_2, \dots, x_n}$ and scalars ${c_1, c_2, \dots, c_n}$, the linear combination is: $c_1x_1 + c_2x_2 + \dots + c_nx_n$.

• The "Lego" Metaphor: Linear algebra fundamentally treats vectors/matrices as building blocks (like Legos). Scalar multiplication and addition are the primary tools used to construct new objects from existing sets.

• Key Analytical Questions:   1. Set Description: What does the set of all possible linear combinations of a given set of objects look like?   2. Possibility: Can a specific given object be constructed from a linear combination of a specific set of building blocks?

• Geometric Representation:   - Single Vector Set: The set of all linear combinations of $(1, 1)$ in $\mathbb{R}^2$ is the line $y = x$ passing through the origin.   

• Two Vector Set: If vectors are not parallel (e.g., $(1, 1)$ and $(-2, 3)$, the set of all linear combinations fills the entire coordinate plane.

• Rephrasing to Systems of Equations: Determining if a vector can be made from a set involves finding scalars $c_1, c_2, c_3$. This is equivalent to solving an augmented matrix. If the reduction leads to a false statement (e.g., $0 = 7$), there is no solution and the combination is impossible.

Open Questions and Future Research for Students

• Matrix Calculations to Ponder: Students should consider how (or if) the following are possible for a matrix $A$:   - Exponents: $A^n$ for integers n > 1 (repeated multiplication).   

• Zero Power/Reciprocals: $A^0$ or $\frac{1}{A}$.   

• Transcendental Functions: What would $e^A$ or $\sin(A)$ look like?   

  • Note: Functions like $\sin(A)$ are generally not obvious without Calculus II principles (Taylor Series).

Section 2.2: Properties of Matrix Operations

• Additive Identity: The zero matrix (denoted $O$ or $o_n$/$o_{m \times n}$). Adding this matrix to any matrix $A$ results in $A$.

• Additive Inverse: The matrix $-A$ such that $A + (-A) = O$. This matrix effectively "undoes" matrix $A$ using addition.

• Multiplicative Identity (The Identity Matrix): Denoted by $I$.   

  • It must be a square matrix.   

  • It contains ones along the main diagonal and zeros everywhere else.   

  • In reduced row echelon form, it contains the maximum number of pivots.  

• Application for Non-Square Matrices: If matrix $A$ is $m \times n$:     

  • The right identity must be $I_n$ ($A \times I_n = A$).     - The left identity must be $I_m$ ($I_m \times A = A$).

The Transpose Operation

• Unary Operation: Unlike multiplication (binary), the transpose requires only one matrix to operate.

• Notation: $A^T$.

• Formal Definition: If matrix $A_{ij}$ is size $m \times n$, then $A^T$ is size $n \times m$ such that the entry at row $j$, column $i$ in $A^T$ was the entry at row $i$, column $j$ in $A$.

• Procedural Visualization:   

  • Method 1: Columns of $A$ become rows of $A^T$. Rows of $A$ become columns of $A^T$.

  • Method 2 ("Twirling"): Imagine the matrix rotating or flipping over its main diagonal axis. Entries on the main diagonal axis do not move, while off-diagonal elements swap places.

Symmetric Matrices

• Definition: A matrix is symmetric if $A^T = A$.

• Conditions:   - All symmetric matrices must be square.   - Elements across the main diagonal must be mirror images.

• Standard Examples: The Identity matrix ($I$) is always symmetric. The zero matrix ($O$) is symmetric only when it is square.

• Generating Symmetric Matrices: For any matrix $B$ (regardless of size), the products $BB^T$ and $B^T B$ will always result in a symmetric matrix.

• Proof of Symmetry for $BB^T$:   - $(BB^T)^T = (B^T)^T B^T$ (Using reversal property of products).   - Since $(B^T)^T = B$, $(BB^T)^T = BB^T$.   

  • Therefore, the product is symmetric.

Properties of the Transpose

1. Double Transpose: $(A^T)^T = A$.

2. Product Rule: $(AB)^T = B^T A^T$ (The order of the matrices must be reversed to maintain valid dimensions).

Nilpotent Matrices: A Case Study

• Definition: A matrix $A$ is nilpotent if $A^m = O$ for some integer $m \ge 2$, provided $A \neq O$.

• Zero Product Property Disparity: In real numbers, if $x \times y = 0$, either $x=0$ or $y=0$. This is not true for matrices. A matrix squared can equal the zero matrix even if the matrix itself is non-zero.

• Problem Formulation: Describe conditions on a $2 \times 2$ matrix $A = \begin{pmatrix} a & b \ c & d \end{pmatrix}$ such that $A^2 = O$.

• Step-by-Step Algebraic Process:   1. Square the matrix: $\begin{pmatrix} a & b \ c & d \end{pmatrix} \begin{pmatrix} a & b \ c & d \end{pmatrix} = \begin{pmatrix} a^2 + bc & ab + bd \ ac + cd & bc + d^2 \end{pmatrix}$ to zero:      

• $a^2 + bc = 0$      

• $b(a + d) = 0$      

• $c(a + d) = 0$      
  $bc + d^2 = 0$   

• 3. Analyze Relationships:      

• $bc = -a^2$      

• $bc = -d^2$      

• Therefore, $a^2 = d^2$, implying $a = \pm d$.   

• 4. Logic Scenarios:      

• If $b \neq 0$ or $c \neq 0$, then the equations $b(a+d)=0$ and $c(a+d)=0$ force $a = -d$.     

  • A valid example of a nilpotent matrix is $\begin{pmatrix} 0 & 3 \ 0 & 0 \end{pmatrix}$.

Section 4.1: Vectors in R^n

• Definition: $\mathbb{R}^n$ refers to the set of $n$-tuples with real entries. Subscripts define the dimensions (e.g., $\mathbb{R}^2$ is 2D, $\mathbb{R}^3$ is 3D).

• Notation Symbols:   - $(3, 4)$ can be written as a column vector $\begin{pmatrix} 3 \ 4 \end{pmatrix}$ or a row vector $\begin{pmatrix} 3 & 4 \end{pmatrix}$.   - Physics/Calc III notation ($3i + 4j + 0k$ or angle brackets $\langle 3, 4 \rangle$) is generally avoided here to prevent confusion with other matrix operations.

• Geometric Representation: Vectors are directed line segments, characterized by a magnitude (length) and a direction.   - Tail: Initial point.   - Head: Terminal point.   - Standard Position: A vector whose tail is specifically at the origin ($0, 0$).

Vector Operations and Special Vectors

• Standard Unit Vectors:   - In $\mathbb{R}^2$: $e_1 = (1, 0)$ and $e_2 = (0, 1)$.   - In $\mathbb{R}^3$: $e_1 = (1, 0, 0)$, $e_2 = (0, 1, 0)$, and $e_3 = (0, 0, 1)$.

• Zero Vector: A point at the origin with length zero and no defined direction.

• Scalar Multiplication ($cu$):   

• If |c| > 1: Stretches the magnitude.  

• If |c| < 1: Compresses the magnitude.   - If c < 0: Flips the direction 180 degrees backwards.

• Vector Addition Geometrics:   - Parallelogram Method: Construct a parallelogram using the two vectors; the diagonal represents the sum.   - Head-to-Tail Method: Place the tail of the second vector at the head of the first; the vector from the starting tail to the final head is the sum.

Section 5.1: Length and the Dot Product in R^n

• Norm (Euclidean Length): Represented by $||u||$.   - Formula: $||u|| = \sqrt{u \cdot u} = \sqrt{u_1^2 + u_2^2 + \dots + u_n^2}$.   - Theorem: $u \cdot u \ge 0$ for all vectors $u$.   - Theorem: $u \cdot u = 0$ if and only if $u$ is the zero vector.

• Normalization: The process of scaling a non-zero vector to have a length (norm) of exactly 1 while maintaining its original direction.   - To normalize vector $u$, multiply by its reciprocal norm: $v = \frac{1}{||u||} u$.   

  • Application: Crucial in Data Science to ensure different metrics (e.g., salary vs. years of education) are on equal footing for comparison without one metric dwarfing the other.

• Normalization Example:   - Vector $u = (3, -2)$.   

  • $||u|| = \sqrt{3^2 + (-2)^2} = \sqrt{9+4} = \sqrt{13}$.   - Normalized Vector: $v = \left(\frac{3}{\sqrt{13}}, \frac{-2}{\sqrt{13}}\right)$.

Questions & Discussion

• Question: Where are the recordings?   - Answer: In the class Zoom meetings section, in the "cloud recordings" tab.

• Question: Can we make the Millennium Falcon with three Legos?   - Answer: No. This was a metaphor for linear combinations—if your "building blocks" (vectors) are limited, you cannot build any arbitrary shape (vector) unless it falls within the span of those blocks.

• Question: Is the dot product associative?   - Answer: No. $(u \cdot v) \cdot w$ is undefined because $(u \cdot v)$ is a scalar, and you cannot "dot" a scalar with a vector. This is a rare instance of an operation not being associative.

• Administrative Reminder: Attendance check was taken via the chat at 9:15 PM.