Matrix derivatives cheat sheet

Vocabulary terms summarizing matrix manipulation rules and common vector derivatives from Kirsty McNaught's 2017 guide.

Transpose of a matrix product ($(AB)^T$)

$B^T A^T$; the order is reversed and everything is transposed.

Transpose of a vector-matrix expression ($(a^T B c)^T$)

$$c^T B^T a$$

Commutativity of the dot product ($a^T b$)

$b^T a$; the result is a scalar, and the transpose of a scalar is itself.

Distributive property of matrix multiplication ($(A + B)C$)

$AC + BC$; multiplication is distributive.

Distributive property with vectors ($(a + b)^T C$)

$$a^T C + b^T C$$

Non-commutativity of matrix multiplication ($AB \neq BA$)

Matrix multiplication is not commutative.

Vector derivative of $x^T B$

$$B$$

Vector derivative of $x^T b$

$$b$$

Vector derivative of $x^T x$

$$2x$$

Vector derivative of $x^T Bx$

$$2Bx$$