Inner Products, Orthogonality and Isometries

What’s the Hermitian inner product?

The Hermitian inner product on $\mathbb{C}^n$ is defined by $\langle x,y\rangle = \sum_{i=1}^n x_i \overline{y_i}$ for $x=(x_1,\dots,x_n),\; y=(y_1,\dots,y_n)\in \mathbb{C}^n.$

What’s an inner product on a vector space V?

A function $V\times V\to\mathbb{F}:(u,v)\rightarrow\langle u,v\rangle$ which satisfies the following:

$$\text{(1) } \langle u,v\rangle = \overline{\langle v,u\rangle}$$

$$\text{(2) } \langle \alpha u + w, v\rangle = \alpha \langle u,v\rangle + \langle w,v\rangle$$

$\text{(3) } \langle v,v\rangle \ge 0 \text{ and } \langle v,v\rangle = 0 \iff v = 0.$"

For all $u,v,w \in V, \alpha \in \mathbb{F}$

What does $\langle u,\alpha v\rangle$ equal when V is an inner product space?

$$\overline{\alpha}\langle u,v\rangle$$

What’s the definition of an orthogonal set?

A set $S\subseteq V\setminus{}\left\lbrace0\right\rbrace$ is orthogonal if, for all $u,v\in S$

$$u\ne v\implies\langle u,v\rangle=0$$

In other words, the set $S$ of non-zero vectors is orthogonal if any two of its elements are othogonal.

Proof sketch: why is any orthogonal set of non-zero vectors linearly independent?

Assume $\lambda_1u_1+\cdots+\lambda_ku_k=0$.

Take the inner product with one vector $u_j$.

Orthogonality kills all the other terms, so we get $\lambda_j\langle u_j,u_j\rangle=0$, i.e. $\lambda_j\|u_j\|^2=0$. Since $u_j\neq0$, we have $\|u_j\|^2\neq0$, so $\lambda_j=0$.

This works for every $j$, so all coefficients are zero. Therefore the set is linearly independent.

What is the Gram-Schmidt process and what are the steps for it.

A way to turn a linearly independent set into an orthonormal set.

The Gram-Schmidt process consists of the following steps.

1. Calculate the set of vectors defined by

   1. $$v_1=u_1$$

   2. $$v_2=u_2-\frac{\langle u_2,v_1\rangle}{\langle v_1,v_1\rangle}v_1$$

   3. $$v_3=u_3-\frac{\langle u_3,v_1\rangle}{\langle v_1,v_1\rangle}v_1-\frac{\langle u_3,v_2\rangle}{\langle v_2,v_2\rangle}v_2$$

   4. $$\vdots$$

   5. $$v_k=u_k-\sum_{i=1}^{k-1}\frac{\langle u_k,v_i\rangle}{\langle v_i,v_i\rangle}v_i$$

2. Normalise the $v_i$ to obtain a set of unit vectors.

   1. $$C=\left\lbrace\frac{1}{\left\Vert v_1\right\Vert}v_1,\ldots,\frac{1}{\left\Vert v_{k}\right\Vert}v_{k}\right\rbrace$$

Remark: If we come across a vector that is difficult to work with due to awkward fractions for example, we can replace it by a non-zero scale factor.

What does it mean for a linear transformation $T:V→W$ , where $V$,$W$ are inner product spaces over the same field, to preserve norms and preserve inner products?

T preserves norms if $\left\Vert T\left(v\right)\right\Vert=\left\Vert v\right\Vert,\forall v\in V$

T preserves inner products if $\langle T\left(v_1\right),T\left(v_2\right)\rangle=\langle v_1,v_2\rangle$

If T preserves norms, what can we say about it preserving inner products?

T preserves norms $\iff$ T preserves inner products

Proof that

• T is an isomorphisms and preserves inner products

• T is surjective and preserves norms

are equivalent for any linear transformation $T:V→W$ where $V,W$ are inner product spaces.

Proof:

($\implies$) Since T is an isomorphism it is surjective and since it preserves inner products it preserves norms.

($\impliedby$) Since T preserves norms, it preserves inner products. To prove T is injective and hence an isomorphism, we show $Ker(T)=\left\lbrace0\right\rbrace$. Let $v$ be such that $T(v)=0$. Then $\left\Vert v\right\Vert=\left\Vert T\left(v\right)\right\Vert=\left\Vert0\right\Vert=0$ so $v=0$. This shows $Ker(T)\subseteq\left\lbrace0\right\rbrace$ . Since $T(0)=0$ we have $0\in Ker(T)$ so $Ker(T)=\left\lbrace0\right\rbrace$.

What is an isometry?

A linear transformation $T:V→W$ where $V,W$ are inner product spaces, is an isometry if $T$ is surjective and preserves norms.

If there is an isometry $V→W$ then we say that $V$ and $W$ are isometric.

Proof sketch: Let $V,W$ be $n$-dimensional inner product spaces and $T:V\to W$ linear. If $\{v_1,\ldots,v_n\}$ is an orthonormal basis of $V$, prove the equivalence between:

(a) $T$ preserves norms,

(b) $\{T(v_1),\ldots,T(v_n)\}$ is an orthonormal basis of $W$, and

(c) $T$ is an isometry.

(a) $\Rightarrow$ (b):

If $T$ preserves norms, then it preserves inner products, so $\langle T(v_i),T(v_j)\rangle=\langle v_i,v_j\rangle$. Since $\{v_1,\ldots,v_n\}$ is orthonormal, the vectors $T(v_1),\ldots,T(v_n)$ are also orthonormal. There are $n$ of them in an $n$-dimensional space $W$, so they form an orthonormal basis.

(b) $\Rightarrow$ (c):

If $\{T(v_1),\ldots,T(v_n)\}$ is an orthonormal basis of $W$, then every $w\in W$ can be written as $w=\sum \mu_iT(v_i)=T(\sum \mu_iv_i)$, so $T$ is surjective. Also, if $v=\sum \lambda_iv_i$, then $T(v)=\sum \lambda_iT(v_i)$. Using orthonormality, $\|v\|^2=\sum|\lambda_i|^2$ and $\|T(v)\|^2=\sum|\lambda_i|^2$, so $\|T(v)\|=\|v\|$. Hence $T$ is an isometry.

(c) $\Rightarrow$ (a): Immediate, since an isometry preserves norms.

What’s the conjugate transpose of a matrix A?

The $n\times m$ matrix $A^*_{nm}(\mathbb{C})$ whose $(i,j)$ entries are defined by $A_{ij}^{*}=\overline{A_{ji}}$.

Properties of the conjugate transpose?

$$(A+B)^*+A^*+B^*$$

$$(\alpha A)^*=\bar{\alpha}A^*$$

$$(BA)^*=A^*B^*$$

where $\alpha$ is a complex number.

What does it mean for a matrix to be unitary?

The columns form an orthogonal basis for $\mathbb{F}^n$. When $\mathbb{F}=\mathbb{R}$ we say $U$ is orthogonal.

$$U^{-1}=U^*$$

Summarise the equivalent conditions for a unitary matrix $U\in M_n(\mathbb F)$.

The following are equivalent:

• $U$ is unitary

• $$U^*U=I_n$$

• $$UU^*=I_n$$

• the rows of $U$, transposed as columns, form an orthonormal basis of $\mathbb F^n$.

• The columns of $U$ form an orthonormal basis of $\mathbb{F}^n$

Proof idea: let $v_1,\ldots,v_n$ be the columns of $U$.

The $(i,j)$ entry of $U^*U$ is $\langle v_j,v_i\rangle$, so $U^*U=I_n$ means the columns of $U$ are orthonormal.

Since there are $n$ orthonormal vectors in $\mathbb F^n$, they form an orthonormal basis.

Similarly, if the rows of $U$ are written as column vectors $u_1,\ldots,u_n$, then the $(i,j)$ entry of $UU^*$ is $\langle u_i,u_j\rangle$, so $UU^*=I_n$ means the rows are orthonormal.

Finally, if $U^*U=I_n$, then $U^*$ is a left inverse of the square matrix $U$, so $U^{-1}=U^*$ and hence $UU^*=I_n$. Thus the conditions are equivalent.

What is the inverse of a unitary matrix $U$?

$$U^*$$