Comprehensive Linear Algebra Notes

Chapter 1 Matrices and Basic Operations

Introduction

• Linear algebra studies sets of matrices and operations on them.
• Matrices help organise coefficients/ constants of linear systems.

Matrices

• A matrix is a rectangular array of numbers written inside brackets/parentheses.
• Order: $m \times n$ means $m$ rows, $n$ columns.
• Entry notation: $a_{ij}$ is element in row $i$ column $j$.
• Special names
  • Row vector: one-row matrix.
  • Column vector: one-column matrix.
  • Square matrix: $n \times n$.
• Real vs complex matrices: entries real ⇒ real matrix, else complex.

Equality (1.3)

• $A=B$ iff same order and every corresponding entry equal.

Addition (1.4)

• $C=A+B$ where $c<em>{ij}=a</em>{ij}+b_{ij}$.
• Laws
  • Commutative: $A+B=B+A$.
  • Associative: $(A+B)+C=A+(B+C)$.
• Zero/Null matrix $0_{m\times n}$: all entries 0; $A+0=A$.
• Negative: $-A$ has entries $-a_{ij}$; $A+(-A)=0$.

Scalar Multiplication (1.5)

• $kA$ multiplies every entry by scalar $k$.
• $1A=A, (-1)A=-A$.

Matrix Multiplication (1.6)

• Defined when columns of $A$ = rows of $B$.
• $AB$ entry: $c<em>{ij}=\sum</em>{k=1}^{n} a<em>{ik}b</em>{kj}.$
• Identity $I_n$: ones on main diagonal, zeros elsewhere. $AI=IA=A$ and unique.
• Laws (selected)
  • Associative: $(AB)C=A(BC)$.
  • Distributive: $A(B+C)=AB+AC$.
  • Scalar: $\alpha(AB)=(\alpha A)B=A(\alpha B)$.
• Non-commutative: generally $AB\neq BA$.

Powers of a Matrix (1.7)

• $A^k=\underbrace{AA\dots A}<em>{k \text{ times}}$, $A^0=I</em>n$.
• Laws: $A^rA^s=A^{r+s},\;(A^r)^s=A^{rs}.$

Idempotent Matrix (1.8)

• $A^2=A.$
• Then $A^n=A$ for $n\ge 1$.

Chapter 2 Transpose of a Matrix

Definition

• $A^T$ obtained by interchanging rows & columns, $(A^T)<em>{ij}=a</em>{ji}$.

Properties

• $(A^T)^T=A$.
• $(\alpha A)^T=\alpha A^T$.
• $(A+B)^T=A^T+B^T$.
• $(AB)^T=B^TA^T$.

Special Square Matrices

• Diagonal: off-diagonal zeros.
• Upper triangular: elements below main diagonal 0.
• Lower triangular: elements above main diagonal 0.
• Symmetric: $A^T=A$.
• Skew-symmetric: $A^T=-A$ (diagonal entries 0).
• Every square matrix $A=S+K$ with $S=\tfrac12(A+A^T)$ symmetric, $K=\tfrac12(A-A^T)$ skew-symmetric.

Chapter 3 Systems of Linear Equations

Terminology

• Linear system in $n$ variables ${x<em>j}$: $\sum</em>{j=1}^{n} a<em>{ij}x</em>j=b_i$ for $i=1,\dots,m.$
• Homogeneous: all $b_i=0$ (always consistent; trivial solution all zeros).

Matrix Form

• $AX=B$ where $A$ coefficient matrix, $X$ variable column, $B$ RHS.
• Augmented matrix $[A\;B]$.

Row Operations (EROs)

1. Swap rows.
2. Multiply row by non-zero scalar.
3. Add multiple of one row to another.

Gaussian Elimination

• Reduce $[A\;B]$ to Row-Echelon (RE) form then solve by back-substitution.
• RE conditions:
  1. Zero rows below non-zero.
  2. Leading entry of each non-zero row to right of row above.
• Reduced Row-Echelon (RRE): additionally leading entries are 1 and column-unique.

Rank (to preview)

• Number of non-zero rows in RE form = rank.

Solution Types

• Unique: rank $=n$.
• Infinite: rank < n and consistent.
• None: inconsistent (e.g. 0=2 row).

Chapter 4 Inverse of a Matrix

Definition

• $A$ invertible if $\exists\,B$ such that $AB=BA=I$; then $B=A^{-1}$ (unique).

Properties

• $(A^{-1})^{-1}=A$.
• $(A^T)^{-1}=(A^{-1})^T$.
• $(kA)^{-1}=k^{-1}A^{-1}$.
• $(AB)^{-1}=B^{-1}A^{-1}$.

2×2 Formula

A=\begin{bmatrix}a&b\c&d\end{bmatrix},\;\det A\neq0\Rightarrow A^{-1}=\frac{1}{ad-bc}\begin{bmatrix}d&-b\-c&a\end{bmatrix}.

Elementary Matrices

• Obtained by applying one ERO to $I_n$.
• Invertible; inverse is elementary.
• Pre-multiplying by elementary matrix ⇔ performing same ERO.

Finding $A^{-1}$ via EROs

• Form $[A\;I] \rightarrow [I\;A^{-1}]$.

Positive & Negative Powers

• For integer $k$, $A^k$ defined using $A^{-1}$ when k<0.

Chapter 5 Determinants

Definitions

• Order 1: $|a|=a$.
• Order 2: \begin{vmatrix}a&b\c&d\end{vmatrix}=ad-bc.
• Order 3 (Sarrus):
  \begin{vmatrix}a1&b1&c1\a2&b2&c2\a3&b3&c3\end{vmatrix}=a1b2c3+b1c2a3+c1a2b3-a3b2c1-b3c2a1-c3a2b_1.
• General $n$: expand along a row/column using minors & cofactors.

Cofactor $A<em>{ij}=(-1)^{i+j}M</em>{ij}$.

Properties (selection)

• Row/col swap multiplies det by $-1$.
• Factor $k$ in a row ⇒ det multiplies by $k$.
• Row addition ((Ri+kRj)) keeps det.
• Zero row/col ⇒ det 0.
• Equal rows ⇒ det 0.
• Triangular: det = product diagonal.
• $|AB|=|A||B|.$

Adjoint & Inverse

• Adjoint $\operatorname{adj} A=(A_{ij})^T$.
• $A\,\operatorname{adj}A=|A|I\Rightarrow A^{-1}=\dfrac{\operatorname{adj}A}{|A|}$ (if $|A|\neq0$).

Cramer’s Rule

• Unique solution ($|A|\neq0$): $x<em>j=\dfrac{|A</em>j|}{|A|}$ where $A_j$ replaces $j$-th column by RHS.

Chapter 6 Linear Independence & Rank

Linear Combination

• $u=c<em>1v</em>1+\cdots+c<em>nv</em>n$.

Independence

• Vectors ${v<em>i}$ independent if $\sum c</em>i v<em>i=0$ only with all $c</em>i=0$; otherwise dependent.

Tests via Matrices

• Arrange vectors as rows or columns; reduce to RE.
• Dependent ⇔ zero row appears.

Rank $r(A)$

• Maximum number of independent rows (equals column rank).
• Equal to non-zero rows in any RE form.
• $r(A)\le \min(m,n).$

System Consistency (Rouché–Capelli)

• AX=B consistent iff $r(A)=r([A\;B])$.
• Number of solutions:
  • Unique: $r=n$.
  • Infinite: r<n.

Chapter 7 Eigenvalues, Eigenvectors & Diagonalization

Definitions

• Eigenvalue $\lambda$: non-zero vector $x$ s.t. $Ax=\lambda x$.
• Eigenvector: associated $x\neq0$.

Characteristic Polynomial

• $p(\lambda)=|A-\lambda I|$.
• Roots are eigenvalues.
• Properties: product of eigenvalues $=|A|$; sum $=\operatorname{tr}A$.

Finding Eigenvectors

• Solve $(A-\lambda I)x=0$ for each eigenvalue.

Similarity & Diagonalization

• $A\sim B$ if $\exists P\, (P^{-1}AP=B).$ Similar matrices share eigenvalues & trace.
• Diagonalizable ⇔ $A\sim D=\operatorname{diag}(\lambda<em>1,\dots,\lambda</em>n).$
• Sufficient condition: $n$ distinct eigenvalues ⇒ diagonalizable.
• General criterion (Theorem): $A$ diagonalizable iff it has $n$ linearly independent eigenvectors; then $P$ formed by these eigenvectors yields $P^{-1}AP=D$.

Cayley–Hamilton Theorem

• Every square matrix satisfies its characteristic polynomial: $p(A)=0$.
• Applications
  • Compute $A^{-1}$ when $|A|\neq0$ by rearranging $p(A)=0$.
  • Express high powers $A^k$ via lower powers.