Linear Algebra Lecture Notes Flashcards

Linear Algebra: Matrix Definitions and Classifications

• Matrix Structure: A matrix is an $m \times n$ arrangement of numbers (elements or entries) organized into $m$ rows and $n$ columns.
• Notation: $A_{m \times n} = [a_{ij}]_{m \times n}$, where $1 \le i \le m$ (row index) and $1 \le j \le n$ (column index).
• Fields: A matrix whose entries are from a field $F$ is known as a matrix over field $F$.
• Square Matrix: A matrix where the number of rows is equal to the number of columns ($n \times n$ or $A_{n \times n}$).
• Principal Diagonal: The line containing elements $a_{ii}$ (where $i = j$). These are called the principal diagonal elements.
• Other Diagonal Lines:
  • Super-diagonal line: Contains elements $a_{i, i+1}$.
  • Sub-diagonal line: Contains elements $a_{i, i-1}$.
• Element Counts in Square Matrices of Order $n$:
  • Total number of principal diagonal elements = $n$.
  • Total number of super-diagonal elements = $n-1$.
  • Total number of sub-diagonal elements = $n-1$.
• Diagonal Matrix: A square matrix in which all non-diagonal elements ($a_{ij}$ where $i \neq j$) are zero. A square null matrix is a diagonal matrix, but a diagonal matrix need not be a null matrix.
• Scalar Matrix: A diagonal matrix where all diagonal elements are identical ($a_{11} = a_{22} = \dots = a_{nn}$).
• Unit Matrix (Identity Matrix): A scalar matrix where the diagonal entries are all equal to $1$.
• Triangular Matrices:
  • Upper Triangular Matrix: A matrix $[a_{ij}]_{n \times n}$ where all elements below the principal diagonal are zero ($a_{ij} = 0$ for all $i > j$).
  • Strictly Upper Triangular: An upper triangular matrix where the diagonal elements are also zero ($a_{ij} = 0$ for all $i \ge j$).
  • Lower Triangular Matrix: A matrix where all elements above the principal diagonal are zero ($a_{ij} = 0$ for all $i < j$).

Submatrices: Definitions and Counting Principles

• Submatrix Definition: A matrix obtained from a given matrix $A$ by deleting any number of rows and columns. Note that deleting zero rows or zero columns results in the matrix itself as a submatrix.
• Counting Submatrices of Order $p \times q$ from Matrix $A_{m \times n}$:
  • If all entries of $A$ are distinct: Total number = $\binom{m}{p} \times \binom{n}{q}$.
  • If all entries of $A$ are the same: Total number = $1$.
• Total Number of Submatrices of Matrix $A_{m \times n}$:
  • If all entries are distinct: Total = $\sum_{p=1}^{m} \sum_{q=1}^{n} \binom{m}{p} \times \binom{n}{q} = (2^m - 1)(2^n - 1)$.
  • If all entries are the same: Total = $m \times n$.
• Numerical Examples for Submatrices:
  • Let $A$ be of order $5 \times 6$.
    • Minimum number of submatrices ($\alpha$ where entries are same) = $5 \times 6 = 30$.
    • Maximum number of submatrices ($\beta$ where entries are distinct) = $(2^5 - 1)(2^6 - 1) = 31 \times 63 = 1953$.
    • Resulting pair $(\alpha, \beta) = (30, 1953)$.
  • Example for 1-rowed submatrices of a $5 \times 6$ matrix: $\alpha = 6$ if entries are same; if distinct, it would be $\binom{5}{1} \times (2^6 - 1)$.
• Consecutive Submatrix: A matrix obtained by taking adjacent (consecutive) rows and columns from the original matrix.
  • If entries are distinct, total consecutive submatrices of order $p \times q$ = $(m-p+1)(n-q+1)$.
  • Total number of all consecutive submatrices = $\frac{m(m+1)}{2} \times \frac{n(n+1)}{2}$.
  • If entries are the same, the total is $1$ for a specific order $p \times q$, and the total distinct consecutive submatrices by content remains based on the dimensions.

Principal Submatrices

• Definition: A square submatrix of a square matrix $A_{n \times n}$ in which the principal diagonal entries are also principal diagonal entries of the original matrix.
• Counting Principal Submatrices of Order $r$:
  • If entries are distinct: Total = $\binom{n}{r}$.
  • If entries are the same: Total = $1$.
• Total Number of Principal Submatrices:
  • If entries are distinct: Total = $\sum_{r=1}^{n} \binom{n}{r} = 2^n - 1$.
  • If entries are the same: Total = $n$.
  • Example: For order $5$ and distinct entries, total = $2^5 - 1 = 31$. If entries are same, total = $5$.

Matrix Operations and Algebraic Properties

• Addition: Two matrices $A$ and $B$ can be added if and only if they have the same order. Addition is commutative: $A + B = B + A$.
• Multiplication by Scalar: $k \cdot [a_{ij}]_{m \times n} = [k a_{ij}]_{m \times n}$.
• Matrix Multiplication: Two matrices $A_{m \times n}$ and $B_{n \times p}$ can be multiplied, resulting in $C_{m \times p}$. Multiplication is generally not commutative ($AB \neq BA$) but is associative: $A(BC) = (AB)C$ and distributive: $A(B+C) = AB + AC$.
• Transpose Properties:
  • $(A+B)^T = A^T + B^T$
  • $(kA)^T = k A^T$
  • $(AB)^T = B^T A^T$ (Reversal Law)
  • $(A^T)^T = A$

Trace of a Matrix

• Definition: The trace of a square matrix $\text{tr}(A)$ is the sum of the principal diagonal elements: $\text{tr}(A) = \sum_{i=1}^{n} a_{ii}$.
• Properties:
  • $\text{tr}(A+B) = \text{tr}(A) + \text{tr}(B)$
  • $\text{tr}(kA) = k \text{tr}(A)$
  • $\text{tr}(AB) = \text{tr}(BA)$
  • $\text{tr}(ABC) = \text{tr}(BCA) = \text{tr}(CAB)$ (Cyclic Permutation).
  • Note: $\text{tr}(ABC)$ is generally not equal to $\text{tr}(ACB)$.
• Theorems on Row and Column Sums:
  • If the sum of elements of each row of matrix $A$ is $\alpha$ and for matrix $B$ is $\beta$, then the sum of elements of each row of the product matrix $AB$ is $\alpha \beta$.
  • Similarly, if the sum of elements of each column of $A$ is $\alpha$ and for $B$ is $\beta$, the column sum of $AB$ is $\alpha \beta$.
  • If each row/column sum is $\alpha$, the total sum of all elements in $A_{n \times n}$ is $n \alpha$.
  • Example: If row sum of square matrix $A$ of order $5$ is $1$, then the row sum of $A^5$ is $1^5 = 1$, and the total sum of all elements of $A^5$ is $5 \times 1 = 5$.

Determinants and Adjoint Matrices

• Determinant Properties:
  • $|AB| = |A| \cdot |B|$
  • $|kA| = k^n |A|$ for order $n$.
  • If two adjacent rows/columns are interchanged, the determinant sign changes.
  • If two rows/columns are identical or linearly dependent (LD), the determinant is $0$.
  • $|A^T| = |A|$
  • For similar matrices ($P M P^{-1} = N$), $|M| = |N|$.
• Minors and Cofactors:
  • Minor ($M_{ij}$): Determinant of the submatrix after deleting the $i$-th row and $j$-th column.
  • Cofactor ($C_{ij}$): $C_{ij} = (-1)^{i+j} M_{ij}$.
  • Determinant calculation: Sum of products of elements of any row/column with their corresponding cofactors.
• Adjoint and Inverse:
  • Adjoint (adj $A$): Transpose of the cofactor matrix.
  • Properties:
    • $A(\text{adj } A) = (\text{adj } A)A = |A|I$
    • $|\text{adj } A| = |A|^{n-1}$
    • $|\text{adj}(\text{adj } A)| = |A|^{(n-1)^2}$
    • $\text{adj}(\text{adj } A) = |A|^{n-2} A$
    • $\text{adj}(AB) = \text{adj}(B) \cdot \text{adj}(A)$
    • $\text{adj}(A^T) = (\text{adj } A)^T$
    • $\text{adj}(\lambda A) = \lambda^{n-1} \text{adj } A$
  • Rank and Adjoint Relation:
    • If $\text{rank}(A) = n$, then $\text{rank}(\text{adj } A) = n$.
    • If $\text{rank}(A) = n-1$, then $\text{rank}(\text{adj } A) = 1$.
    • If $\text{rank}(A) < n-1$, then $\text{rank}(\text{adj } A) = 0$.
  • Inverse: $A^{-1} = \frac{\text{adj } A}{|A|}$, provided $|A| \neq 0$ (Non-singular).

Matrix Conjugation and Hermitian Properties

• Conjugate ($\bar{A}$): A matrix where every element is replaced by its complex conjugate ($a + ib \rightarrow a - ib$).
• Tranjugate ($A^{\theta}$): Transpose of the conjugate matrix ($A^{\theta} = (\bar{A})^T$).
• Hermitian Matrix: $A^{\theta} = A$. Principal diagonal elements must be real.
• Skew-Hermitian Matrix: $A^{\theta} = -A$. Principal diagonal elements are either zero or purely imaginary.
• Unitary Matrix: $A A^{\theta} = I$.
  • Properties: $\text{det}(A)$ has modulus equal to $1$. All orthogonal matrices are unitary matrices (real entries only).

Symmetric and Orthogonal Matrices

• Symmetric Matrix: $A^T = A$.
• Skew-Symmetric Matrix: $A^T = -A$. Principal diagonal elements are all zero.
  • The determinant of a skew-symmetric matrix of odd order is always $0$.
• Orthogonal Matrix: $A A^T = I = A^T A$.
  • Property: Sum of squares of elements in any row/column is $1$. Sum of products of corresponding elements of two distinct rows/columns is zero.
  • The determinant of an orthogonal matrix is always $\pm 1$.

Specialized Square Matrices

• Idempotent Matrix: $A^2 = A$.
  • If $A$ and $B$ are idempotent and $AB = BA$, then $AB$ is idempotent.
  • If $AB = BA = 0$, then $A+B$ is idempotent.
• Involutory Matrix: $A^2 = I$.
  • Characteristic: $(A+I)(A-I) = 0$.
• Nilpotent Matrix: $A^k = 0$ for some positive integer $k$. The smallest such $k$ is the index of the matrix.
  • Property: Index of nilpotency is always $\le$ the order of the matrix. Trace and determinant of a nilpotent matrix are zero.

Rank of a Matrix

• Definition: The largest order of a non-zero minor.
• Computation via Echelon Form: Rank equals the number of non-zero rows in the row echelon form obtained via elementary row transformations (ERT).
• Equivalent Matrices: Two matrices $A$ and $B$ are equivalent if one is obtained from the other by finite elementary transformations ($B = PAQ$, where $P$ and $Q$ are non-singular). Equivalent matrices have the same rank.
• Normal Form: A matrix transformed into the block form $\begin{pmatrix} I_r & 0 \\ 0 & 0 \end{pmatrix}$ has rank $r$.
• Rank Properties:
  • $\text{rank}(A+B) \le \text{rank}(A) + \text{rank}(B)$.
  • $\text{rank}(AB) \le \min(\text{rank}(A), \text{rank}(B))$.
  • $\text{rank}(AB) \ge \text{rank}(A) + \text{rank}(B) - n$ (Sylvester's Law of Nullity).
  • $\text{rank}(A^T) = \text{rank}(A) = \text{rank}(A A^T)$.
  • Rank of skew-symmetric matrix cannot be $1$.

Systems of Linear Equations

• Form: $AX = B$, where $[A|B]$ is the augmented matrix.
• Consistency:
  • If $\text{rank}(A) \neq \text{rank}(A|B)$, the system has no solution (Inconsistent).
  • If $\text{rank}(A) = \text{rank}(A|B) = n$ (number of variables), the system has a unique solution.
  • If $\text{rank}(A) = \text{rank}(A|B) < n$, the system has infinite solutions.
• Homogeneous Systems ($AX = 0$):
  • Always consistent ($X=0$ is the trivial solution).
  • Non-trivial solutions exist if $\text{rank}(A) < n$.

Eigenvalues and Eigenvectors

• Characteristic Equation: $|A - \lambda I| = 0$. Solutions for $\lambda$ are the eigenvalues.
• Eigenvector: A non-zero vector $X$ such that $AX = \lambda X$.
• Algebraic Multiplicity (AM): The number of times an eigenvalue repeats in the characteristic equation.
• Geometric Multiplicity (GM): The number of linearly independent eigenvectors associated with an eigenvalue: $\text{GM}(\lambda) = n - \text{rank}(A - \lambda I)$.
  • Relationship: $1 \le \text{GM}(\lambda) \le \text{AM}(\lambda)$.
• Properties:
  • Sum of eigenvalues = $\text{tr}(A)$.
  • Product of eigenvalues = $|A|$.
  • Cayley-Hamilton Theorem: Every square matrix satisfies its own characteristic equation. It can be used to find $A^{-1}$.
  • Minimal Polynomial: The smallest degree monic polynomial that annihilates matrix $A$. It divides the characteristic polynomial.

Groups, Fields, and Vector Spaces

• Binary Operation: A function mapping $S \times S \rightarrow S$ (Internal) or $F \times V \rightarrow V$ (External).
• Group: A set with a binary operation satisfying closure, associativity, identity existence, and inverse existence. If commutative, it is an Abelian group.
• Field: A set with two operations (addition and multiplication) where it is an abelian group under addition, the set excluding zero is an abelian group under multiplication, and multiplication is distributive over addition.
  • Example: $\mathbb{R}$, $\mathbb{C}$, and $\mathbb{Q}$ are fields. Integers $\mathbb{Z}$ do not form a field because non-zero elements lack multiplicative inverses in $\mathbb{Z}$.
• Vector Space: A set of vectors $V$ over a field $F$ satisfying $10$ axioms involving vector addition (Abelian group) and scalar multiplication properties (distributivity, associativity of scalars, scalar unity).