Matrices Notes

Matrices

Basic Arithmetic of Matrices

Definition 7.1: A matrix is a rectangular array of numbers. Rows are horizontal units, and columns are vertical units.
- The element in the $i$ th row and $j$ th column is the $(i, j)$ th entry of the matrix.
- An $m \times n$ matrix has $m$ rows and $n$ columns.
- Rows are counted from the top, columns from the left.
- The collection of all $m \times n$ matrices with real entries is denoted by $M_{m\times n}(\mathbb{R})$ .
- Capital letters denote matrices, small letters denote entries. For example, $A = (a{ij}){m\times n}$ denotes an $m \times n$ matrix $A$ with entries $a_{ij}$ , where $1 \leqslant i \leqslant m$ and $1 \leqslant j \leqslant n$ .
Example 7.1: The matrix $A = \begin{pmatrix} 1 & 2 & 3 \ 4 & 5 & 6 \end{pmatrix}$ is a $2 \times 3$ matrix. Its first row is $\begin{pmatrix} 1 & 2 & 3 \end{pmatrix}$ , and its third column is $\begin{pmatrix} 3 \ 6 \end{pmatrix}$ . The $(2, 1)$ th entry of $A$ is 4.
Definition 7.2: A square matrix has the same number of rows and columns.
Definition 7.3: The main diagonal of a matrix consists of the $(1, 1)$ th, $(2, 2)$ th, …, $(n, n)$ th entries.
Definition 7.4: A diagonal matrix is a square matrix whose entries not on the main diagonal are all 0.
Example 7.2:
- The matrix $A = \begin{pmatrix} 4 & 0 & 0 \ 0 & 5 & 0 \ 0 & 0 & 0 \end{pmatrix}$ is a diagonal matrix (and thus also a square matrix).
- The matrix $B = \begin{pmatrix} 0 & 0 & 1 \ 0 & 2 & 0 \ 3 & 0 & 0 \end{pmatrix}$ is a square matrix but not a diagonal matrix.
Definition 7.5: Two matrices $A = (a{ij}){m\times n}$ and $B = (b{ij}){k\times \ell}$ are equal if $m = k$ , $n = \ell$ , and $a{ij} = b{ij}$ for all $i$ and $j$ . We write $A = B$ if $A$ and $B$ are equal matrices.
Definition 7.6: Let $A = (a{ij}){m\times n}$ and $B = (b{ij}){m\times n}$ be two matrices of the same size. The matrix addition is defined by $A + B = (a{ij} + b{ij})$ .
Example 7.3: $\begin{pmatrix} 1 & 2 \ 3 & 4 \end{pmatrix} + \begin{pmatrix} 5 & 7 \ 8 & 6 \end{pmatrix} = \begin{pmatrix} 1+5 & 2+7 \ 3+8 & 4+6 \end{pmatrix} = \begin{pmatrix} 6 & 9 \ 11 & 10 \end{pmatrix}$ .
Definition 7.7: The $m \times n$ zero matrix, denoted by $O_{m\times n}$ or simply $O$ , is the $m \times n$ matrix where every entry is 0.
Definition 7.8: Let $A = (a{ij}){m\times n}$ be a matrix. The additive inverse of $A$ , denoted by $-A$ , is the matrix $(-a{ij}){m\times n}$ .
Proposition 7.1: Let $A$ , $B$ , and $C$ be matrices of the same size, and let $O$ be the zero matrix of the same size. Then the following hold:
- (a) (commutativity) $A + B = B + A$
- (b) (associativity) $(A + B) + C = A + (B + C)$
- (c) $O + A = A$
- (d) $A + (-A) = O$
Definition 7.9: Let $A = (a{ij}){m\times n}$ and $B = (b{ij}){m\times n}$ be two matrices of the same size. The matrix subtraction is defined by $A - B = (a{ij} - b{ij})$ .
Example 7.4: $\begin{pmatrix} 5 & 7 \ 8 & 6 \end{pmatrix} - \begin{pmatrix} 1 & 2 \ 3 & 4 \end{pmatrix} = \begin{pmatrix} 5-1 & 7-2 \ 8-3 & 6-4 \end{pmatrix} = \begin{pmatrix} 4 & 5 \ 5 & 2 \end{pmatrix}$ .
Definition 7.10: Let $A = (a{ij}){m\times n}$ be a matrix and let $c$ be a scalar. The scalar multiplication is defined by $cA = (ca_{ij})$ .
Example 7.5: $5 \begin{pmatrix} 1 & 2 \ 3 & 4 \end{pmatrix} = \begin{pmatrix} 5(1) & 5(2) \ 5(3) & 5(4) \end{pmatrix} = \begin{pmatrix} 5 & 10 \ 15 & 20 \end{pmatrix}$ .
Proposition 7.2: Let $A$ and $B$ be matrices of the same size, and let $c$ and $d$ be two scalars. Then the following hold:
- (a) $1(A) = A$ , $0(A) = O$ , $(-1)(A) = -A$
- (b) $(cd)A = c(dA)$
- (c) (distributivity) $c(A + B) = cA + cB$ , $(c + d)A = cA + dA$
Definition 7.11: Let $A = (a{ij}){m\times n}$ be a matrix. The transpose of $A$ , denoted by $A^T$ , is the matrix $(a{ji}){n\times m}$ .
Example 7.6: $\begin{pmatrix} 1 & 2 \ 3 & 4 \ 5 & 6 \end{pmatrix}^T = \begin{pmatrix} 1 & 3 & 5 \ 2 & 4 & 6 \end{pmatrix}$ .
Proposition 7.3: Let $A$ and $B$ be matrices of the same size, and let $c$ be a scalar. Then the following hold:
- (a) $(A^T)^T = A$
- (b) $(A + B)^T = A^T + B^T$
- (c) $(cA)^T = cA^T$
Definition 7.12: Let $A = (a{ij}){m\times n}$ and $B = (b{ij}){n\times \ell}$ be matrices. The matrix product $AB$ of $A$ and $B$ is defined to be the matrix $(c{ij}){m\times \ell}$ where $c{ij} = a{i1}b{1j} + a{i2}b{2j} + \cdots + a{in}b_{nj}$ .
The product $AB$ is only defined when the number of columns of $A$ is the same as the number of rows of $B$ .
We can define the positive power of a square matrix in the natural way. For example, $A^2 = AA$ and $A^3 = (A^2)A$ , etc.
Example 7.7: $\begin{pmatrix} 1 & 2 \ 3 & 4 \end{pmatrix} \begin{pmatrix} 5 & 6 \ 7 & 8 \end{pmatrix} = \begin{pmatrix} (1)(5)+(2)(7) & (1)(6)+(2)(8) \ (3)(5)+(4)(7) & (3)(6)+(4)(8) \end{pmatrix} = \begin{pmatrix} 19 & 22 \ 43 & 50 \end{pmatrix}$ .
Definition 7.13: The $n \times n$ identity matrix, denoted by $I_n$ or simply $I$ , is the $n \times n$ diagonal matrix whose every diagonal entry is 1.
Proposition 7.4: Let $A$ , $B$ , and $C$ be matrices such that the following operations are well-defined, and let $c$ be a scalar. Then the following hold:
- (a) (associativity) $(AB)C = A(BC)$
- (b) $AO = O$ , $OB = O$ , $AI = A$ , $IB = B$
- (c) (distributivity) $A(B + C) = AB + AC$ , $(A + B)C = AC + BC$
- (d) $c(AB) = (cA)B = A(cB)$
- (e) $(AB)^T = B^T A^T$
Matrix multiplication is not commutative in general. This means $AB$ and $BA$ are not necessarily the same even if both are well-defined.
Definition 7.14: An $n \times n$ matrix $A$ is said to be invertible or non-singular if there exists an $n \times n$ matrix $B$ such that $AB = I_n = BA$ . Such a matrix $B$ is called the inverse of $A$ , and is usually denoted by $A^{-1}$ . A square matrix which has no inverse is said to be non-invertible or singular.
Proposition 7.5: The inverse of a matrix, if it exists, is unique.
Example 7.8: The matrix $A = \begin{pmatrix} 1 & 2 \ 3 & 5 \end{pmatrix}$ is invertible, and $A^{-1} = \begin{pmatrix} -5 & 2 \ 3 & -1 \end{pmatrix}$ .
Example 7.9: The matrix $B = \begin{pmatrix} 1 & 2 \ 3 & 6 \end{pmatrix}$ is non-invertible.
Proposition 7.6: Let $A$ and $B$ be invertible matrices of the same size, and let $c$ be a nonzero scalar. Then the following hold:
- (a) $A^{-1}$ is invertible and $(A^{-1})^{-1} = A$
- (b) $cA$ is invertible and $(cA)^{-1} = c^{-1}A^{-1}$
- (c) $AB$ is invertible and $(AB)^{-1} = B^{-1}A^{-1}$
- (d) $A^T$ is invertible and $(A^T)^{-1} = (A^{-1})^T$

Determinants

Definition 7.15: The determinants of a $1 \times 1$ matrix $A = (a)$ , a $2 \times 2$ matrix $B = \begin{pmatrix} a & b \ c & d \end{pmatrix}$ , and a $3 \times 3$ matrix $C = \begin{pmatrix} a & b & c \ d & e & f \ g & h & i \end{pmatrix}$ are defined to be $\det A = a$ , $\det B = ad - bc$ , $\det C = aei + bfg + cdh - gec - dbi - ahf$ respectively.
- The determinant of $A$ can also be represented as $|A|$ . Another common notation for the determinant of a matrix $\begin{pmatrix} a & b \ c & d \end{pmatrix}$ is $\begin{vmatrix} a & b \ c & d \end{vmatrix}$ .
Example 7.10: $\begin{vmatrix} 1 & 2 & 0 \ 3 & 4 & 5 \ 0 & 6 & 7 \end{vmatrix} = (1)(4)(7) + (2)(5)(0) + (0)(3)(6) - (0)(4)(0) - (3)(2)(7) - (1)(6)(5) = -44$ . Also, we have $\det O = 0$ and $\det I = 1$ .
Theorem 7.1: A square matrix $A$ is invertible if and only if $\det A \neq 0$ .
Example 7.11: Since $\begin{vmatrix} 3 & 2 \ 6 & 4 \end{vmatrix} = 0$ , the matrix $\begin{pmatrix} 3 & 2 \ 6 & 4 \end{pmatrix}$ is non-invertible.
Proposition 7.7: Let $A$ and $B$ be square matrices of size $n \times n$ . Then the following hold:
- (a) $\det(AB) = (\det A)(\det B)$
- (b) $\det(kA) = k^n \det A$ for any $k \in \mathbb{R}$
- (c) $\det(A^T) = \det A$
- (d) if $A$ is invertible, then $\det(A^{-1}) = \frac{1}{\det A}$
- (e) if $A$ has a zero row or a zero column, then $\det A = 0$
- (f) if $A$ has two identical rows or two identical columns, then $\det A = 0$
Example 7.12: Let both $A$ and $B$ be $3 \times 3$ matrices such that $\det A = 2$ and $B$ is invertible. Then $\det(-B^T A^2 B^{-1}) = -4$ .
Definition 7.16: Let $A = (a{ij})$ be an $n \times n$ matrix where $n \geqslant 2$ . Let $A{ij}$ be the $(n - 1) \times (n - 1)$ matrix obtained by deleting the $i$ th row and the $j$ th column of $A$ . The $(i, j)$ -cofactor of $A$ is defined by $c{ij} = (-1)^{i+j} \det(A{ij})$ .
Example 7.13: The $(2, 3)$ -cofactor of $\begin{pmatrix} 1 & 2 & 3 \ 4 & 5 & 6 \ 7 & 8 & 9 \end{pmatrix}$ is $-\begin{vmatrix} 1 & 2 \ 7 & 8 \end{vmatrix} = 6$ .
Theorem 7.2: Let $A$ be an invertible matrix. Let $B$ be the matrix whose $(i, j)$ th entry is the $(j, i)$ -cofactor $c_{ji}$ of $A$ . Then $A^{-1} = \frac{1}{\det A} B$ .
Example 7.14: Let $A = \begin{pmatrix} a & b \ c & d \end{pmatrix}$ where $\det A = ad - bc \neq 0$ . Then $A^{-1} = \frac{1}{ad - bc} \begin{pmatrix} d & -b \ -c & a \end{pmatrix}$ .
Example 7.15: The inverse of $B = \begin{pmatrix} 1 & 1 & 2 \ 2 & 3 & 4 \ 1 & 0 & 1 \end{pmatrix}$ is $\begin{pmatrix} -3 & 1 & 2 \ -2 & 1 & 0 \ 3 & -1 & -1 \end{pmatrix}$ .

Links

Theorems
- 7.1: Criterion for a matrix to be invertible
- 7.2: Inverse of a matrix
Propositions
- 7.1: Properties of matrix addition
- 7.2: Properties of scalar multiplication
- 7.3: Properties of transposes
- 7.4: Properties of matrix multiplication
- 7.5: Uniqueness of the matrix inverse
- 7.6: Properties of matrix inverses
- 7.7: Properties of determinants
Terminologies and Notations
- $M_{m\times n}(\mathbb{R})$
- additive inverse $-A$
- cofactor $c_{ij}$
- column
- determinant $\det A, |A|$
- diagonal matrix
- entry
- equal (matrix)
- identity matrix $I$
- inverse (matrix) $A^{-1}$
- invertible matrix
- main diagonal
- matrices
- matrix
- matrix addition $A + B$
- matrix power $A^k$
- matrix product $AB$
- matrix subtraction $A - B$
- non-invertible matrix
- non-singular
- row
- scalar multiplication $cA$
- singular
- square matrix
- transpose $A^T$
- zero matrix $O$